Stakeholder 
and  Process 
Alignment 

in  Navy  Installation 
Technology  Transitions 


Adventures  in  Defense  Acquisition 

by  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics 


Bridging  the 
“Valley  of  Death” 


EVM  System’s 
High  Cost 

Fact  or  Fiction? 


A  Practical  PM 
Guide  to  Requests  for 
Equitable  Adjustment 


r~T  11H 


From  the  Under  Secretary  of  Defense  for  Acquisition 
Technology,  and  Logistics 

Adventures  in  Defense  Acquisition 

Frank  Kendall 


J  LJ 


3GD 


i 


Stakeholder  and  Process  Alignment 
in  Navy  Installation  Technology  Transitions 

Eva  Regnier,  Ph.D.,  Robert  W.  Barron ,  Ph.D.,  Daniel  A.  Nussbaum, 
Pb.D.r  and  Kail  Macias 

Technology  Transition  Programs  (TTPs)  are  important  tools  for 
facilitating  technology  transfer  from  science  and  technology 
development  to  DoD  operational  adoption. 


Bridging  the  "Valley  of  Death" 

Anthony  Davis  and  Tom  Ballenger 

The  U.S.  Special  Operations  Command  has  developed  a  new 
methodology  to  facilitate  technology  transition  that  other  acquisi¬ 
tion  organizations  might  consider  using. 


EVM  System's  High  Cost 
Fact  or  Fiction? 

(First  in  a  series  of  articles) 

Ivan  Bembers,  Ed  Knox,  Michelle  Jones 
and  Jeff  Traczyk 

It  has  been  suggested  that  the  government 
can  do  away  with  EVMS  in  favor  of  more 
efficient  and  affordable  management 
techniques.  Is  there  a  basis  for  this? 


A  Practical  PM  Guide  to  Requests 
for  Equitable  Adjustment 

Lt  Col  Scott  Klempner,  USAF 
A  Request  for  Equitable  Adjustment 
resulted  from  a  technical  disagreement 
between  the  contractor  and  the  govern¬ 
ment  about  how  much  in-scope  testing 
was  required  to  properly  resolve  a  space¬ 
craft  test  fault.  The  contractor  persistently 
brought  up  the  issue. 


Getting  the  Capabilities  Right 
The  Next  Generation  of  Air  Force 
CBRN  Protection 

Alexander  R.  Slate  and 
Maj  Charles  O'Connor,  USAF 
Bulky  and  thermally  burdensome  chemi¬ 
cal,  biological,  radiological  and  nuclear 
protective  garments  have  become  the 
norm,  along  with  protective  masks  that 
greatly  restrict  vision  and  head  mobility. 


The  Seven  Lethal 
Acquisition  Diseases 

Brian  Schultz 

Poor  team  behaviors  that  should  be 
avoided  are  one  of  the  seven  lethal  acqui¬ 
sition  diseases.  High-performing  teams 
provide  potential  remedies. 


Requirements  Management 
The  Need  to  Overhaul  JCIDS 

Thomas  H.  Miller 

Two  pieces  of  the  "big  'A' "  acquisition 
process— the  requirements  and  the 
financial  systems— have  received  less 
congressional  attention  than  the  Defense 
Acquisition  System.  Yet  these  processes 
have  contributed  to  continuing  poor 
acquisition  results  in  various  areas. 


Take  a  Deep  Dive  With  DAU 
The  Kind  We  Offer  Is  Safe  and 
Sure  to  Keep  You  Dry 

Woody  Spring  and 
Rebecca  Haydu-Jackson 
As  part  of  its  Mission  Assistance  pro¬ 
gram,  DAU  has  conducted  deep  dives 
for  more  than  a  decade.  These  in-depth 
analyses  of  our  customers'  organizations 
facilitate  understanding  the  inherent 
challenges  (and  the  root  causes  thereof). 


Redefining  the  "Can  Do"  Attitude 
The  Language  of  a  Critical 
Thinking  Culture 

Christopher  Phillips ,  Ph.D. 
and  Bobbie  DeLeon ,  Ed.D. 

Sticking  with  existing  procedures,  come 
what  may,  a  reluctance  to  question 
authority,  and  buying  into  "group  men¬ 
tality"  are  all  contributors  to  potential 
disasters.  Critical  thinking  and  risk 
management  require  rethinking  habitual 
patterns  and  developing  an  openness  to 
new  approaches. 


ALSO 


Auditing  Organizational 
Security 

Eugene  A.  Razzetti 


P  P  OSD  Logistics  Fellowship 
J  J  —A  View  From  Above 

Bryan  L.  Jerkatis 

OC  MDAP/MAIS  Program 
Manager  Changes 


Defense 

AT&L 

Vo  I  XLVI  No.  1,  DAU  254 


Published  by  the 

DEFENSE  ACQUISITION  UNIVERSITY 


Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics 

Frank  Kendall 

DAU  President 

James  P.  Woolsey 

DAU  Chief  of  Staff 

Joseph  Johnson 

Director,  DAU  Operations  Support  Group 

Leo  Filipowicz 

Director,  DAU  Visual  Arts  and  Press 

Randy  Weekes 

Defense  AT&L  Editorial  Staff 

Managing  Editor/Senior  Editor,  DAU  Press 

Benjamin  Tyree 

Art  Director 

Tia  Gray 


Online  Content  Editor 

Collie  J.  Johnson 

Production  Manager 

Frances  Battle 

Online  Support 

Nina  Austin 


Copy  Editor/ 
Circulation  Manager 

Debbie  Gonzalez 

Editorial  Support 

Noelia  Gamboa 
Michael  Shoemaker 


Article  preparation/submission  guidelines  are  located 
on  the  inside  back  cover  of  each  issue  or  may  be  down¬ 
loaded  from  our  website  at  <http://www.dau.mil/pub- 
scats/pages/defenseatl.aspx>.  Inquiries  concerning 
proposed  articles  can  be  made  by  e-mail  to  datl@dau. 
mil  or  by  phone  to  703-805-4282  or  DSN  655-4282. 

Subscribe/unsubscribe/change  of  address:  Fill  out,  sign, 
and  fax  or  e-mail  the  subscription  form  in  the  back  of 
this  issue,  or  download  the  form  at  <http://dau.dodlive. 
mil/files/2015/04/Online-Subscription.pdf>. 

Privacy  Act/Freedom  of  Information  Act 

If  you  provide  us  your  business  address,  you  will  be¬ 
come  part  of  mailing  lists  that  are  public  information 
and  may  be  provided  to  other  agencies  upon  request. 
If  you  prefer  not  to  be  part  of  these  lists,  use  your  home 
address.  Do  not  include  your  rank,  grade,  service,  or 
other  personal  identifiers. 

Defense  AT&L  (ISSN  1547-5476),  formerly  Program 
Manager,  is  published  bimonthly  by  the  DAU  Press  and  is 
free  to  all  U.S.  and  foreign  national  subscribers.  Periodical 
postage  is  paid  at  the  U.S.  Postal  Facility,  Fort  Belvoir,  Va., 
and  additional  U.S.  postal  facilities. 

POSTMASTER,  send  address  changes  to: 

DEFENSE  AT&L 

DEFENSE  ACQUISITION  UNIVERSITY 
ATTN  DAU  PRESS  STE  3 
9820  BELVOIR  ROAD 
FT  BELVOIR  VA  22060-5565 

Disclaimer 

Statements  of  fact  or  opinion  appearing  in  Defense 
AT&L  are  solely  those  of  the  authors  and  are  not  nec¬ 
essarily  endorsed  by  the  Department  of  Defense,  the 
Office  of  the  Under  Secretary  of  Defense,  Acquisition, 
Technology  and  Logistics,  or  the  Defense  Acquisition 
University.  Articles  are  in  the  public  domain  and  may  be 
reprinted  or  posted  on  the  Internet.  When  reprinting, 
please  credit  the  author  and  Defense  AT&L. 

Some  photos  appearing  in  this  publication  may  be  digi¬ 
tally  enhanced. 


Defense  AT&L:  January-February  2017 


FROM  THE  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISITION,  TECHNOLOGY,  AND  LOGISTICS 


Adventures  in  Defense  Acquisition 

Frank  Kendall 


For  what  is  likely  to  be  my  last  commu¬ 
nication  to  the  acquisition  workforce 
as  Under  Secretary  of  Defense  for  Ac¬ 
quisition,  Technology,  and  Logistics 
(USD[AT&L]),  I  thought  I  would  share 
with  you  a  few  stories,  all  true,  from  my  45 
or  so  years  working  in  various  aspects  of  de¬ 
fense  acquisition,  either  in  uniform,  as  a  civil 
servant,  in  industry,  or  as  an  appointee.  I've 
put  them  more  or  less  in  chronological  order, 
starting  with  an  experience  I  had  while  serv¬ 
ing  in  Europe  during  the  height  of  the  Cold 
War.  There  has  certainly  been  a  lot  of  water 
under  the  bridge  since  then,  and  a  lot  has 
changed,  but  the  things  I've  learned  along 
the  way  are  in  many  cases  timeless. 

During  the  1970s,  as  an  Army  captain,  I  commanded  a  Hawk 
air  defense  battery  in  West  Germany.  We  had  a  new  battal¬ 
ion  commander  take  over  during  that  time.  He  immediately 
started  a  program  he  called  "Victory  Through  Integrity"  or 
VTI.  This  was  the  period  of  the  readiness  crisis  and  the  "hollow 
force"  following  the  end  of  the  war  in  Vietnam. 

Our  new  commander's  ideas  on  logistics  included  that  cross¬ 
leveling  parts  between  units  and  cannibalizing  down  items  of 
equipment,  like  our  radars,  was  a  violation  of  our  integrity. 
We  stopped  doing  these  things  and  went  nonoperational  for 
several  months  while  we  stubbornly  stuck  to  our  "principles" 
about  these  maintenance  policies.  During  that  period,  train¬ 
ing  as  well  as  operational  readiness  suffered  enormously. 
Eventually,  the  battalion  commander  was  told  to  change  his 
policies.  He  very  reluctantly  obeyed  the  order.  I  believe  it  is 
always  important  to  act  in  a  principled  way,  and  in  particular 
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to  act  with  integrity,  but  in  this  case  I  felt  that  my  commander 
had  confused  integrity  with  reasonable  choices  in  manage¬ 
ment  policy.  Leaders  will  always  have  initiatives  and  labels 
to  describe  them  (e.g.,  Better  Buying  Power),  but  when  they 
represent  management  choices  they  should  be  viewed  as  just 
that— choices  that  can  be  reversed  or  changed  based  on  new 
information  (data)  about  how  well  they  are  working,  or  not. 

In  1980,  while  still  an  Army  captain,  I  attended  my  first  con¬ 
gressional  hearing.  I  believe  it  was  the  House  Armed  Services 
Committee.  I  was  there  in  support  of  my  boss  at  the  time, 
the  Army  major  general  who  was  the  Army's  Ballistic  Missile 
Defense  program  manager.  He  was  one  in  a  series  of  program 
managers  providing  testimony  that  day.  This  was  about  3  years 
before  President  Reagan  announced  the  Strategic  Defense  Ini¬ 
tiative  (SDI)  program. 

One  of  my  most  vivid  memories  of  that  hearing  was  the  lead 
professional  staff  member  for  the  committee  holding  up  a 
schedule  and  chastising  a  witness  for  the  degree  of  concur¬ 
rency  in  his  program.  What  I  can't  remember  is  whether  he 
was  for  or  against  concurrency— but,  whichever  it  was,  he 
was  passionate  about  it.  We've  been  for  and  against  con¬ 
currency  several  times  since  that  hearing.  Like  many  other 
decisions,  the  degree  of  concurrency  (overlap  between  devel¬ 
opment  and  production)  in  a  program  is  a  judgment  call  moti¬ 
vated  by  many  factors,  first  among  them  being  confidence  in 
the  stability  of  the  design.  Early  in  my  tenure  as  USD(AT&L), 
I  referred  to  the  extraordinary  amount  of  concurrency,  and 
the  specific  decision  to  start  production  on  the  F-35  fighter 
jet  before  any  flight  test  data  had  been  accumulated,  as  "ac¬ 
quisition  malpractice.''  The  press  loves  pithy  expressions  like 
this,  so  the  comment  got  a  lot  of  exposure.  Concurrency  deci¬ 
sions,  like  many  others  in  acquisition,  require  critical  thinking, 
sound  professional  judgment  and  taking  a  lot  of  program 
specific  factors  into  account. 

Careers  can  take  strange  turns.  One  of  mine  may  have  hinged 
on  a  2  a.m.  flight  from  Andrews  Air  Force  Base  in  Maryland  to 
Nantucket  Island  in  Massachusetts.  I  was  the  Assistant  Dep¬ 
uty  Director  of  Defense  Research  and  Engineering  for  Strategic 
Defense  programs.  My  boss'  boss'  boss,  the  Under  Secretary 
for  Acquisition,  was  on  vacation  in  Nantucket  and  was  tasked 
on  short  notice  to  come  back  to  Washington  for  a  hearing  on 
the  SDI.  I  volunteered  to  fly  to  Nantucket  on  the  Ml  LAI  R  flight 
that  would  bring  him  back  to  DC  and  to  prep  him  during  the 
flight  for  the  hearing,  which  would  be  held  the  same  day.  We 
picked  him  up  at  about  5  a.m.  Nobody  had  told  him  I  would  be 
on  the  airplane,  so  he  was  a  little  surprised  to  see  me.  He  was 
also  pretty  impressed  that  I  had  gone  the  extra  mile  to  stay  up 
all  night  so  I  could  brief  him.  I  accompanied  him  to  the  hear¬ 
ing,  which  went  very  well,  in  part  because  I  had  a  chance  to 


■4 


DH  I 


Leaders  will  always  have  initiatives 
and  labels  to  describe  them  (e.g.. 
Better  Buying  Power),  but  when 
they  represent  management 
choices  they  should  bejviewed 
as  just  that— choices  tmt  can  be 


reversed  or  changed  based  on  new 


information  (data)  about  how  well  ;  r* 
ay  are  working,  or  not. 


prep  him  thoroughly.  Just  after  that,  I  applied  to  be  the  acting 
Director  of  Tactical  Warfare  Programs  when  the  incumbent 
left  government.  This  job,  overseeing  all  of  the  Department  of 
Defense  (DoD)  conventional  weapons  system  programs  and 
reporting  directly  to  the  Under  Secretary,  was  my  dream  job 
at  the  time.  I  got  the  job. 

While  I  was  still  the  acting  Director  for  Tactical  Warfare  Pro¬ 
gram,  a  period  of  2.5  years  when  I  didn't  know  if  a  political 
appointee  would  replace  me,  there  were  four  changes  in  the 
officeholder  of  Under  Secretary  of  Defense  for  Acquisition. 
One  of  these  was  a  former  executive  from  Ford  who  was  totally 
new  to  Washington  and  DoD  and  who  had  just  come  onboard. 
At  the  time,  we  were  struggling  to  get  the  Advanced  Medium- 
Range  Air-to-Air  Missile  program  (AMRAAM)  through  testing 
and  into  production.  Late  on  a  Friday  afternoon,  I  received  a 
preliminary  report  from  the  Air  Force  that  we  had  experienced 
a  flight  test  failure.  There  was  very  little  information  on  what 
had  happened,  so  I  decided  to  wait  until  I  knew  more  before 
informing  the  Under  Secretary.  On  Monday  morning,  I  was 
at  Patuxent  River  Naval  Air  Station  in  Maryland,  getting  a 
medical  so  I  could  do  an  F-18  flight  out  to  a  carrier.  A  perk  of 
my  position  was  that  there  were  often  good  reasons  for  me 
to  experience  firsthand  the  performance  of  our  conventional 
weapons  programs. 

Just  as  the  flight  physician  was  about  to  take  my  blood  pres¬ 
sure,  I  received  a  call  from  the  Under  Secretary.  The  press  had 
heard  about  the  flight  test  failure  and  had  asked  the  Secretary 
of  Defense  about  it.  He  was  clueless,  so  he  asked  the  Under 
Secretary,  who  was  also  clueless  because  I  hadn't  informed 
him  yet.  When  asked,  the  Air  Force  was  understandably  quick 
to  point  out  that  I  had  been  informed  right  after  the  failure. 
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The  Under  Secretary  proceeded  to  rip  me  a  new  one,  as  they 
say.  As  soon  as  I  got  off  the  phone,  the  flight  physician  took 
my  blood  pressure.  Eventually  I  did  get  to  experience  the  F-18 
flight,  and  eventually  the  "acting”  status  was  removed  from 
my  title,  but  it  took  some  time  to  recover  from  that  initial  im¬ 
pression.  Nobody  likes  surprises,  and  the  more  senior  one  is 
the  less  one  likes  them.  Bad  news  does  not  improve  with  age. 

In  addition  to  having  problems  completing  flight  test,  the 
AMRAAM  struggled  for  at  least  a  year  to  demonstrate  that 
it  could  meet  one  specific  reliability  requirement,  the  average 
number  of  hours  it  could  be  carried  on  an  aircraft  before  a 
failure  occurred.  The  requirement  had  been  set  arbitrarily  at 
450  hours.  This  was  a  totally  unrealistic  number  that  later 
analysis  showed  had  no  operational  value  or  cost  effective¬ 
ness.  The  requirement  could  have  been  dropped  to  250  with 
minimal  cost  or  operational  impact.  So  why  did  we  spend 
more  than  a  year  making  holes  in  the  sky  to  prove  we  could 
achieve  450  hours?  Because  we  had  failed  operational  test¬ 
ing  and  it  had  politically  become  a  high-interest  item.  The 
program  had  a  bad  reputation  and  was  at  real  risk  of  cancella¬ 
tion.  The  Services  concluded  that  it  was  better  to  keep  flying 
to  try  to  achieve  the  requirement  than  to  take  the  political 
risk  associated  with  reducing  it;  so  we  kept  flying.  In  those 
days,  requirements  were  often  set  by  relatively  junior  people 
with  a  high  degree  of  arbitrariness.  The  missile  AMRAAM 
was  replacing  had  a  mean  time  between  failures  of  200  flight 
hours.  So  what  was  a  good  number  for  the  replacement? 
How  about  450  hours?  Seemed  reasonable.  Acquisition  and 
operational  people  have  to  work  in  close  cooperation.  If  you 
don't,  this  is  the  sort  of  thing  that  happens. 


One  of  my  programs  in  DoD  was  a  special  access  Navy 
program  to  develop  the  A-12  stealthy  fighter  bomber.  It  had 
already  started  Engineering  and  Manufacturing  Develop¬ 
ment  when  it  fell  under  my  portfolio.  It  was  also  touted  as 
a  new  model  for  how  to  do  acquisition  effectively  at  the 
time— little  oversight,  firm  fixed-price  development,  an  ac¬ 
quisition  approach  that  in  the  development  phase  teamed 
two  competitors  who  would  later  compete  for  production, 
and  a  very  aggressive  schedule  tied  to  fixed-price  produc¬ 
tion  options.  It  was  a  disaster  waiting  to  happen.  The  A-12  is 
taught  as  a  classic  case  study  in  how  not  to  do  acquisition, 
and  for  good  reasons. 

We  have  a  lot  of  programs  that  struggle  to  get  through  devel¬ 
opment  and  into  production,  but  most  of  them  do  get  there. 
Programs  like  the  A-12,  where  we  spend  billions  of  dollars  and 
get  nothing,  are  travesties.  I  won't  try  to  tell  this  whole  story 
here;  it  is  available  elsewhere  in  great  detail.  At  that  time,  the 
Secretary  of  Defense  was  Dick  Cheney,  and  we  were  doing 
something  called  "The  Major  Aircraft  Review.”  In  one  of  my 
briefings  to  Secretary  Cheney,  I  had  told  him  that  based  on 
earned  value  data  (but  not  what  the  contractor  or  military 
Service  were  saying)  the  program  was  in  big  trouble,  and 
would  overrun  by  at  least  a  year  and  $1  billion.  I  found  that 
out  from  the  DoD  Earned  Value  Management  guru  at  the 
time,  Gary  Christie. 

After  the  A-12  blew  up,  figuratively  speaking,  and  was  can¬ 
celed  (properly  so,  as  the  Supreme  Court  finally  concluded 
about  20  years  later)  there  was  an  investigation,  led  by  a 
general  officer,  into  who  knew  what  when.  It  turned  out  that 
John  had  briefed  a  member  of  my  staff  several  weeks  earlier, 
but  no  one  had  informed  me.  The  data  provided  compelling 
evidence  of  where  the  program  was  headed.  That  member 
of  my  staff  who  had  been  briefed  was  a  very  capable  Navy 
officer.  However,  instead  of  informing  me  of  the  data,  he  had 
immediately  called  the  Navy  staff  to  warn  them  about  this 
threat  to  the  Navy's  program  in  the  Office  of  the  Secretary 
of  Defense.  I  was  rather  upset  when  I  found  out  he  hadn't 
seen  any  reason  to  inform  me,  the  person  he  was  supposed 
to  be  working  for.  During  the  investigation,  I  brought  this  up, 
and  in  the  report  that  followed  I  was  criticized  for  not  having 
adequately  trained  this  officer  in  the  fact  that  he  had  a  duty  to 
inform  me,  his  supervisor,  of  any  relevant  information  about 
the  program  he  was  overseeing  for  me.  I'm  not  making  this 
up.  Service  loyalties  run  deep. 

The  A-12  cancellation  came  about  in  part  because  the  Secre¬ 
tary  of  Defense  had  testified  that  the  program  was  progressing 
more  or  less  on  track.  I  don't  know  for  a  fact,  but  my  guess 
is  that  he  simply  forgot  about  the  concerns  I  had  expressed 
to  him  during  the  major  aircraft  review.  He  had  no  reason  to 
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dissemble,  and  he  was  put  on  the  spot  by  a  question  he  had 
not  anticipated.  A  few  months  later,  the  contractors  requested 
a  bailout,  embarrassing  the  Secretary,  who  subsequently  or¬ 
dered  the  program  canceled.  Two  people  on  the  Secretary's 
staff  argued  against  cancellation— me  and  the  Director  of 
Acquisition  Policy,  Eleanor  Spector.  Our  new  boss,  who  re¬ 
placed  the  previous  Under  Secretary  for  Acquisition  at  about 
that  time,  listened  to  us  but  kept  his  cards  close.  The  decision 
meeting  with  Secretary  Cheney  took  place  early  one  morning, 
and  neither  Eleanor  nor  I  attended.  A  few  hours  later,  another 
member  of  the  acquisition  staff,  who  had  been  in  the  Secre¬ 
tary's  briefing  room  for  a  subsequent  meeting,  dropped  off 
a  hard  copy  of  a  set  of  briefing  charts  he  had  found  at  the 
podium.  They  were  the  charts  my  boss,  the  new  Under  Secre¬ 
tary  for  Acquisition,  had  used  to  brief  the  Secretary.  The  final 
chart  read:  Recommendation— Termination.  I  don't  know  to 
this  day  if  that  was  the  right  decision  or  not.  Most  of  the  time, 
as  Eleanor  and  I  maintained,  one  is  better  off  working  through 
problems  to  get  the  needed  capability.  This  isn't  always  the 
case,  however.  I  do  know  that  25  years  later  the  Navy  still 
doesn't  have  a  stealthy  tactical  aircraft  operating  from  a  car¬ 
rier,  but  we  are  getting  close. 

The  Advanced  Self-Protection  Jammer  or  ASPJ  is  another  pro¬ 
gram  that  didn't  make  it  through  the  transition  from  develop¬ 
ment  to  production  and  fielding.  ASPJ  was  another  product  of 
the  fad  of  fixed-price  development  that  was  tried  in  the  late 
1980s.  A  good  deal  of  my  time  in  the  early  1990s  was  spent 
cleaning  up  the  many  messes  that  this  policy  created.  I  have 
good  experience-based  reasons  for  wanting  to  avoid  fixed- 
price  development.  ASPJ  had  another  problem,  however,  and 
it  had  to  do  with  algebra. 

ASPJ  was  a  jamming  system  for  tactical  aircraft.  Its  job  was 
in  part  to  jam  enemy  air  defenses  so  that  tactical  aircraft 
wouldn't  be  shot  down.  In  order  to  get  through  the  Operational 
Testing  phase  to  transition  to  full-rate  production,  ASPJ  had  to 
demonstrate  that  it  could  adequately  perform  this  function. 
The  metric  for  success  was  expressed  as  an  algebraic  equa¬ 
tion  that  had  to  be  statistically  tested.  The  equation  was  built 
in  part  around  the  success  of  the  jammer  at  defeating  a  threat 
after  an  air  defense  missile  was  launched  against  the  aircraft 
with  ASPJ  on  board.  We  made  the  mistake  of  not  including  the 
cases  in  which  ASPJ  was  effective  at  preventing  the  launch, 
so  these  successes  didn't  count  as  part  of  the  test.  Again,  we 
found  ourselves  in  a  situation  where  changing  the  rules  would 
have  been  viewed  with  suspicion  in  the  political  environment 
around  struggling  acquisition  programs.  In  this  case,  we  did 
make  the  needed  changes,  but  for  other  reasons  the  program 
was  canceled  in  the  defense  drawdown  that  followed  the  Cold 
War.  It  was  later  resurrected  with  a  different  name  and  ulti¬ 
mately  fielded. 


For  the  last  few  years,  I  have  been 
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A  few  years  later,  I  had  taken  a  position  at  Raytheon  as  Cor¬ 
porate  Vice  President  of  Engineering.  We  were  in  a  tight  com¬ 
petition  with  our  most  ferocious  competitor,  Hughes  Aircraft, 
to  build  the  next  generation  short-range  air-to-air  missile,  the 
AIM-9X.  We  thought  we  had  a  much  better  design  than  our 
competitor  and  were  sure  we  could  offer  the  customer  much 
better  operational  performance.  We  had  a  problem,  however. 
From  what  we  could  tell  from  the  draft  request  for  proposals 
we  had  seen  and  from  discussions  with  the  Air  Force,  there 
was  no  way  our  higher  performance  could  be  considered  in 
the  source  selection.  We  also  anticipated  a  price  disadvantage 
because  our  missile  design,  though  innovative,  was  more  com¬ 
plex— and  we  believed  more  costly  as  a  result. 

I  spent  a  lot  of  time  in  the  Pentagon  trying  to  get  the  program 
management,  the  operational  community,  or  the  Under  Secre¬ 
tary  for  Acquisition  to  provide  some  way  for  our  better  opera¬ 
tional  performance  (a  bigger  engagement  envelope  and  higher 
probability  of  kill)  to  be  considered  in  source  selection.  I  failed. 
In  this  case,  we  lost— but  this  occurred  just  as  Raytheon  was 
buying  Hughes.  Hughes  had  bid  very  low;  we  speculated  that 
this  was  done  so  Hughes  could  book  the  business  to  enhance 
its  attractiveness  as  an  acquisition.  In  some  respects,  this  was 
a  lowest  price  technically  acceptable  source  selection,  some¬ 
thing  many  in  industry  complain  about  today  and  something  I 
have  tried  to  limit  to  cases  where  it  is  really  appropriate.  Most 
of  the  time  we  do  want  higher  performance,  if  it  is  at  a  price 
we  would  consider  reasonable.  For  the  last  few  years,  I  have 
been  encouraging  or  directing  the  military  Services  to  provide 
bidders  with  a  monetized  adjustment  in  source  selection  as 
a  means  of  encouraging  innovation  and  obtaining  best-value 
solutions.  After  several  examples,  it  is  clear  that  this  approach 
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is  working.  I  wish  it  had  been  used  in  the  1990s  when  we  were 
bidding  on  AIM-9X. 

While  I  was  in  industry,  I  served  for  some  time  on  the  Army 
Science  Advisory  Board.  One  study  we  were  involved  in  was 
a  review  of  a  weapon  system  that  had  featured  prominently 
in  the  First  Gulf  War.  It  happened  to  be  a  weapon  system  that 
my  company  produced.  I  don't  recall  the  reason,  but  as  part  of 
the  study  we  needed  some  technical  data  on  the  system's  per¬ 
formance.  For  reasons  we  didn't  understand,  we  just  couldn't 
get  the  program  office  to  give  us  the  data,  despite  several 
requests.  Finally,  one  of  the  study  group's  members,  retired 
Gen.  Jack  Vessey,  the  former  Vice  Chief  of  Staff  of  the  Army, 
called  the  Chief  of  Staff  to  ask  for  some  help.  We  got  the  data. 
I,  however,  got  a  call  through  my  corporate  headquarters  to  go 
to  Washington  to  meet  with  a  brigadier  general  on  the  Army 
staff  responsible  for  the  program  to  explain  my  reasoning,  as 
I  was  associated  with  the  request.  The  program  office,  fear¬ 
ing  it  might  look  bad  somehow,  had  been  slow-rolling  us  in 
providing  the  data  and  I  was  being  called  to  task  for  having 
gone  over  everyone's  head  to  the  Chief  of  Staff  through  Gen. 
Vessey.  My  management  wasn't  pleased.  Corporations  know 
where  their  money  comes  from,  and  sometimes  the  people 
who  control  those  funds  have  narrow  ideas  of  what  is  right 
and  what  is  wrong. 

Another  incident  from  my  time  in  industry  involved  what  I  can 
only  describe  as  abuse  of  power  by  a  government  acquisition 
official.  At  the  time,  my  firm  had  two  matters,  totally  unrelated 
and  involving  two  programs,  that  we  wanted  resolved  by  the 
Service  in  question.  One  was  a  protest  of  a  bid  we  had  lost.  It 
was  not  at  all  common  for  my  firm  to  protest.  We  felt  that  it 
would  upset  our  customers  and  that  it  was  unlikely  to  succeed. 
In  this  case,  we  had  lost  a  bid  on  something  we  considered  a 
core  business— a  share  of  the  market  and  a  product  that  we 
had  controlled  for  a  very  long  time.  We  felt  we  had  a  legitimate 
reason  to  protest  the  source  selection  and  it  was  important 
business— so  we  protested. 

The  other  matter  was  a  request  we  made  of  the  same  Service 
on  another  program  that  was  coming  up  for  source  selection. 
We  wanted  some  changes  to  the  request-for-proposal  lan¬ 
guage,  changes  we  felt  were  fair  and  that  just  happened  to 
be  to  our  advantage.  With  these  two  matters  on  the  table,  we 
were  visited  by  a  senior  flag  officer  from  the  Service  involved. 
He  asked  us  which  of  the  two  matters  was  most  important 
to  us  and  told  us  that  the  Service's  decisions  on  them  were 
"linked."  I  was  shocked.  In  my  view,  then  and  now,  the  govern¬ 
ment  should  be  resolving  disputes  or  issues  with  industry  on 
a  case-by-case  basis  on  the  merits.  I  never  found  out  if  this 
conduct  was  illegal,  but  I'm  certain  that  it  was  unethical.  The 
government  should  not  cut  backroom  deals  in  which  it  coerces 


a  contractor  to  give  up  a  legal  right  to  a  decision  on  the  merits 
in  return  for  a  competitive  advantage.  The  government  has 
immense  power  over  contractors,  and  has  an  obligation  to 
not  abuse  that  power.  When  it  does  abuse  its  power,  trust  is 
destroyed.  By  the  way,  my  colleagues  from  industry  and  I  did 
exactly  the  right  thing:  We  ignored  the  question. 

While  I  was  in  industry,  I  spent  several  years  as  an  independent 
consultant.  One  of  the  projects  I  participated  in  was  the  Army's 
Future  Combat  Systems  program  or  FCS.  Like  A-12,  this  pro¬ 
gram  wasted  billions  of  dollars  and  delivered  basically  nothing 
to  the  Army.  It  was  hugely  ambitious,  driven  by  a  "vision"  that 
was  divorced  from  reality  and  hobbled  by  totally  unrealistic 
direction  on  schedule,  imposed  from  the  top  of  the  Army. 

The  acquisition  community  within  the  Army  took  huge  risks 
trying  to  execute  the  unrealistic  4.5-year  schedule  from  start 
of  development  to  a  production  decision— for  the  largest  and 
most  complex  program  in  the  history  of  the  DoD.  The  acquisi¬ 
tion  strategy  risks,  including  the  contracting  approach,  a  Lead 
System  Integration,  the  immaturity  of  the  requirements  and 
the  early  loss  of  competitive  incentive  doomed  the  program 
before  it  started.  The  sanity  check  that  the  Under  Secretary 
for  Acquisition  is  supposed  to  provide  failed  under  Service 
pressure  to  proceed.  As  soon  as  the  responsible  leadership 
departed  the  Army,  the  schedule  was  slipped  4  more  years— 
but  the  damage  had  already  been  done.  This  is  the  most 
extreme  example  of  something  I  have  seen  too  many  times; 
operational  and  Service  leadership  is  always  in  a  hurry  and 
usually  has  no  real  understanding  of  what  it  takes  to  design, 
prototype,  test  or  produce  a  specific  product.  This  mistake 
cost  the  Army  more  than  $10  billion  of  precious  research  and 
development  funds  and  several  years  of  modernization  that 
can  never  be  recovered. 

The  Services  do  have  distinct  cultures,  and  that  includes  how 
they  relate  to  outside  stakeholders  and  authorities.  The  clas¬ 
sic  allusion  to  "the  dumb,  the  devious,  and  the  defiant"  isn't 
wholly  accurate,  but  there  are  times  when  it  seems  apt.  A 
better  characterization  might  be  that  the  Army  knows  how 
to  salute  to  a  fault,  the  Air  Force  likes  to  cite  Office  of  the 
Secretary  of  Defense  (OSD)  direction  as  to  why  it  is  acting  in 
a  certain  way,  and  the  Navy  would  strongly  prefer  that  there 
be  no  OSD  direction.  That  is  certainly  an  oversimplification, 
but  it  is  a  rough  approximation  of  reality. 

I  could  tell  stories  all  day  from  my  current  tenure  as  USD(ATScL) 
about  the  Services,  but  here  is  one  from  my  tenure  as  Principle 
Deputy  Under  Secretary:  For  some  reason,  we  were  having  a 
meeting  in  my  office  with  a  brigadier  general  from  the  Army's 
acquisition  community.  We  got  into  a  discussion  of  several 
options  for  how  to  proceed  on  a  specific  program.  It  wasn't 
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a  decision  meeting,  and  staff  members  were  just  tossing  out 
ideas  for  discussion.  We  did  this  for  about  20  minutes  and 
the  meeting  broke  up.  About  an  hour  later,  I  received  a  note 
from  the  Army  Acquisition  Executive  complaining  about  all 
the  direction  the  brigadier  had  been  given.  He  walked  out  the 
room  convinced  that  he  had  just  been  directed  to  do  every 
one  of  the  things  that  had  been  discussed,  when  in  fact  he  had 
been  directed  to  do  none  of  them.  Apparently,  he  went  back 
to  his  office  with  his  hair  on  fire  and  started  ranting  about  all 
the  crazy  guidance  he  was  getting  from  every  member  of  the 
DoD  acquisition  staff.  I'm  guessing  that  the  Air  Force  would 
have  picked  any  guidance  they  liked  and  implemented  it  but 
made  clear  it  was  at  the  direction  of  the  OSD.  The  Navy  would 
probably  have  regarded  it  as  an  amusing  conversation  and 
largely  ignored  it.  Try  as  I  might,  I  don't  know  that  I  ever  con¬ 
vinced  the  Services,  at  least  at  the  program  manager  level,  to 
not  take  direction  from  random  staff  members  with  no  direc¬ 
tive  authority  over  them.  My  policy  was  that  the  staff  was 
there  to  advise  me  as  the  Defense  Acquisition  Executive,  not 
to  provide  direction  to  the  Services— but  implementing  that 
policy  isn't  as  easy  as  it  should  be.  A  program  manager  try¬ 
ing  to  get  his  program  approved  just  wants  it  approved,  and 
is  likely  to  err  on  the  side  of  accepting  direction  if  he  or  she 
thinks  it  will  help  achieve  the  goal.  I  finally  directed  my  staff  to 
identify  all  comments  on  Service  plans  as  "Defense  Acquisi¬ 
tion  Board  Issue,"  discretionary,  or  administrative.  This  meant 
that  I  would  have  visibility  into  anything  the  staff  thought  was 
important  to  change.  I  think  this  has  helped,  but  there  is  still 
room  for  progress. 

We  spend  a  lot  of  time  trying  to  devise  acquisition  strategies 
that  will  effectively  incentivize  industry  to  deliver  more  of 
whatever  the  government  wants.  Industry  has  two  priorities. 
In  order  of  importance,  they  are  to  (1)  win  contracts,  and  (2) 
make  money  on  them.  The  first  is  a  prerequisite  to  the  sec¬ 
ond.  Government  people  should  never  lose  sight  of  the  fact 
that  these  imperatives  always  motivate  industry.  We  can  use 
them  to  get  better  results,  but  we  need  to  be  careful  about 
unintended  consequences. 

A  case  in  point  was  the  Joint  Advanced  Guided  Missile  or 
JAGM,  an  Army-led  joint  program.  The  Army  was  conduct¬ 
ing  a  competition  and  had  asked  industry  to  build  competing 
prototypes  as  risk  reduction  efforts  in  support  of  the  compe¬ 
tition.  The  prototypes  were  to  be  flight  tested  as  part  of  the 
source  selection.  I  had  challenged  the  Army's  intention  to  use 
a  fixed-price  incentive  contract  for  the  next  phase  of  work— 
Engineering  and  Manufacturing  Development.  My  concern 
was  the  degree  of  risk  for  the  upcoming  phase.  I  asked  the 
Army  to  bring  in  the  engineers  for  the  program  to  walk  me 
through  both  competitors'  designs,  the  one  they  would  use 
in  the  early  prototype  testing  as  part  of  the  source-selection 
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process  and  the  production  prototypes  they  were  proposing 
to  actually  build  in  the  next  phase.  What  I  discovered  was 
that  there  was  no  traceability  between  the  risk  reduction  pro¬ 
totypes  and  the  production  prototypes.  Every  subsystem  of 
the  missiles  would  have  to  be  redesigned.  The  competitors 
were  building  "proof  of  principle"  prototypes  for  the  source 
selection.  They  were  not  reducing  the  risk  in  the  designs  they 
intended  to  build  for  production. 


As  a  result  of  this,  I  directed  the  Army  to  change  the  contract 
type  to  one  more  suited  for  the  remaining  risk.  Probably  more 
importantly,  the  light  bulb  went  on  about  what  the  competi¬ 
tors  were  trying  to  do.  They  were  not  motivated  to  reduce 
risk.  That  would  have  entailed  taking  some  risk,  and  that  was 
the  opposite  of  what  they  were  motivated  to  do.  They  were 
motivated  to  win,  which  meant  that  they  wanted  a  low-risk  and 
successful  flight  test  so  that  they  could  win  the  contract.  The 
government  had  asked  for  the  things  our  policy  supports  and 
the  Congress  expects:  competitive  prototypes  and  flight  tests. 
The  government  failed  to  insist  on  prototypes  with  designs 
traceable  to  the  designs  being  bid  for  production  and  to  the 
reduction  of  the  specific  risks  associated  with  those  designs. 
We  can't  blame  industry  for  responding  to  the  business  incen¬ 
tives  we  provide.  The  government  acquisition  team  must  have 
the  expertise  it  needs  to  understand  what  is  required,  and  the 
professionalism  to  ensure  that  industry  provides  it.  Industry 
will  always  act  to  maximize  its  return,  and  the  government  will 
get  what  it  accepts. 


It  has  been  a  great  honor  to  have  led  the  terrific  men  and 
women  in  the  DoD's  acquisition  workforce.  You  are  unsung 
heroes  who,  with  equally  dedicated  and  patriotic  people  in 
industry,  provide  our  men  and  women  in  uniform  with  the 
products  and  services  they  need  to  defend  our  freedom.  I  hope 
that  some  of  these  anecdotes  will  prove  useful  as  you  continue 
your  efforts  to  improve  even  more  on  the  great  work  you  do 
every  day.  Thank  you.  It  has  been  wonderful  to  have  been  part 
of  this  team.  & 
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‘echnology  Transition  Programs  (TTPs)  are 
an  important  tool  for  facilitating  technology 
transfer  from  science  and  technology  (S&T) 
development  to  operational  adoption  in  the 
Department  of  Defense  (DoD).  TTPs  for 
weapons  systems  and  platforms  have  formal  processes  to  smooth  and 
speed  the  path  to  operational  adoption.  By  contrast,  for  technologies  tar¬ 
geted  at  installations,  there  are  some  special  challenges  in  formalizing  the 
transition  process.  This  article  outlines  some  of  the  TTPs  currently  being 
used  in  the  DoD  and  proposes  a  general  framework  for  adapting  their  best 
practices  to  the  larger  TTP  community. 


Regnier  is  associate  professor  of  decision  science  at  the  Naval  Postgraduate  School  (NPS).  She  researches  and 
teaches  decision  making  in  Department  of  Defense  contexts.  Barron  is  a  postdoctoral  fellow  at  the  Center  for  Next 
Generation  Photovoltaics.  His  research  interests  include  technological  change  and  technology  adoption.  Nussbaum 
chairs  the  Energy  Academic  Group  at  the  NPS.  His  research  and  teaching  interests  include  decision  analysis  and  risk 
assessment.  Macias  is  technical  director  for  the  Naval  Facilities  Engineering  and  Expeditionary  Warfare  Center ; 
responsible  for  developing  technology  processes,  research  partnerships ,  and  warfare  center  technical  capabilities. 
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If  a  new  technology  is  adopted, 
it  does  not  necessarily 
replace  every  instance  of 
an  old  technology,  and  the 
installation  may  need  the 
capability  to  maintain  both 
new  and  legacy  equipment  for 
an  extended  period. 
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Stakeholder  and  Process  Alignment  Needed 

The  goal  of  TTPs  is  to  speed  the  development  of  existing  and 
emerging  technologies  for  use  in  defense  applications,  and  to 
increase  the  speed  and  likelihood  of  their  successful  and  cost- 
effective  operational  adoption.  A  successful  technology  must 
not  only  perform  properly,  it  must  interface  smoothly  with 
other  systems,  meet  requirements,  be  appealing  to  end-users 
and  other  stakeholders,  and  be  compatible  with  the  organiza¬ 
tion's  processes,  including  planning,  budgeting,  contracting 
and  technical  approvals.  For  example,  an  energy-saving  tech¬ 
nology  that  has  a  net  operational  cost  savings  over  5  years  may 
still  be  unable  to  compete  with  other  projects  that  support 
operational  requirements  such  as  a  hangar  modernization  or 
an  upgraded  pier. 

Alternatively,  a  high-performance  technology  may  not  be 
adoptable  simply  because  the  Unified  Facilities  Criteria  (UFC) 
doesn't  allow  it.  If  any  of  the  key  stakeholders,  including  main¬ 
tenance,  safety  or  cybersecurity  personnel,  has  a  technical  or 
operational  objection,  the  technology  may  not  be  adopted. 
These  hurdles  really  do  stop  adoption.  As  stated  in  the  2013 
Navy  Environmental  Sustainability  Development  to  Integration 
(NESDI)  program  report: 

We  have  numerous  technical  success  stories  that  are  not  fully 
integrated  because  of  certain  circumstances  or  conditions— 
some  of  which  are  totally  outside  the  realm  of  a  Principal  Inves¬ 
tigator.  However,  these  circumstances  or  conditions  must  be 
identified  so  the  appropriate  person(s)  can  take  action.  Imple¬ 
mentation  of  technology  is  difficult  so  you  need  to  have  a  road¬ 
map  in  place  at  the  start  and  ask  for  directions  along  the  way. 

Stakeholder  and  Process  Alignment  in  TTPs 

The  Government  Accountability  Office  (GAO)  identified  the 
need  for  a  gated  review  process  to  smooth  the  path  for  tech¬ 
nologies  transitioning  from  S&T  into  "product  development" 
in  2006.  By  2013,  GAO  reported  that  many  DoD  TTPs  used 
technology  transition  agreements,  which  call  for  (nonbind¬ 
ing)  commitment  from  stakeholders  as  a  prerequisite  for  a 


technology  moving  through  gates  in  the  program.  Well-known 
TTPs  such  as  the  Joint  Capability  Technology  Demonstration 
program,  and  the  Technology  Insertion  Program  for  Savings, 
target  existing  acquisition  programs,  generally  a  specific  weap¬ 
ons  system  or  platform. 

In  the  installation  environment,  the  need  for  TTPs  to  facilitate 
transition  is  similar,  but  the  adoption  ecosystem  is  different. 
One  of  the  biggest  differences  between  weapons  systems  and 
the  installation  environment  is  that  the  end-users  are  a  more 
diffuse  group.  This  complicates  the  problem  in  a  number  of 
ways.  For  example,  every  installation  has  a  diverse  group  of 
facilities  in  terms  of  age,  use  and  systems,  where  some  may 
benefit  from  reduced  power  consumption  and  peak  loads 
more  than  others,  for  example  by  realizing  more  cost  savings. 

Examples  of  TTPs  for  installation  technologies  in  the  Navy 
are  the  NESDI  program,  the  Navy  Shore  Energy  Technology 
Transition  and  Integration  program,  and  the  Energy  Systems 
Technology  Evaluation  Program  .  Each  of  these  programs  has  a 
multigate  project  review  process  to  ensure  early  consideration 
of  stakeholder  needs  and  administrative  processes. 

General  Framework  for  Stakeholder 
and  Process  Alignment 

In  the  facilities  context,  end-users  and  operational  engineers, 
such  as  installation  energy  managers,  are  rarely  involved  in 
S&T  and  therefore  are  not  in  a  position  to  influence  these  proj¬ 
ects.  Within  S&T,  single-source  procurement,  installation  of 
software,  and  hiring  may  all  face  much  less  stringent  require¬ 
ments  than  in  the  operational  community.  Consequently,  S&T 
engineers  may  not  be  aware  of  real  barriers  to  adoption  facing 
operational  engineers.  Moreover,  TTPs  in  the  weapons  system 
and  platform  acquisition  context,  usually  designate  a  person 
to  line  up  stakeholders  and  approval  authorities,  but  in  the 
installation  context  this  responsibility  often  falls  to  the  S&T 
engineers,  who  may  not  be  properly  trained  to  accomplish  it. 

Here  we  present  the  Adoption  Readiness  Level  (ARL)  scale 
(Table  1)  as  a  tool  for  S&T  engineers  to  facilitate  stakeholder 
and  process  alignment.  The  ARLs  synthesize  and  generalize 
principles  for  managing  development  of  technologies  for  wide¬ 
spread  adoption  in  an  installation  environment. 

In  keeping  with  the  need  to  simultaneously  integrate  tech¬ 
nologies  with  existing  systems,  secure  stakeholder  support 
and  integrate  with  shore  planning  and  funding,  program  and 
process  requirements,  the  ARLs  measure  progress  across 
three  domains:  technology  integration,  stakeholders  and 
processes.  Hurdles  in  any  one  of  these  domains  will  prevent 
adoption,  regardless  of  the  technology's  suitability  in  the  other 
two.  Formal  documentation  and  milestones  related  to  all  three 
areas  increase  the  likelihood  that  important  barriers  will  be 
recognized  and  addressed  before  they  substantially  delay  or 
even  prevent  adoption.  Conversely,  insurmountable  barriers 
that  will  ultimately  prevent  adoption  will  be  recognized  sooner, 
minimizing  the  costs  associated  with  the  failed  project. 
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Table  1.  Summary  of  Adoption  Readiness  Levels 


ARl 

Component 
Technology  TRl 

Systems-Level 

Technology  Integration 

Stakeholders 

Processes 

1 

Application 

Identified 

5 

Potential  to  satisfy  an  exist¬ 
ing  or  anticipated  need  more 
effectively  than  alternatives. 

N/A 

N/A 

Demonstration 

Planning 

5 

Research  plan  developed, 
necessary  facilities  identi¬ 
fied. 

Stakeholders  identified. 
Need  verified. 

Funding  budgeted  for  demonstration 
phase.  Approvals  required  for  demon¬ 
stration  identified. 

3 

Representative 

Prototype 

6 

Demonstrated  at  represen¬ 
tative  research  site.  Perfor¬ 
mance  documented. 

Pilot  performance  vali¬ 
dated  by  stakeholders. 

Technical  approvals  required  for  opera¬ 
tional  use  identified  and  documented. 
Testing  or  modification  requirements 
documented. 

4 

Representative 

Demonstration 

7 

O&S  requirements  and  any 
training  requirements  for 

O&S  documented. 

O&S  funding  levels  and 
personnel  requirements 
for  sustainable  support  in 
operation  estimated. 

Process  for  getting  technical  approvals 
for  operational  use  has  been  docu¬ 
mented. 

5 

Fully  Adoptable 

8 

Operating  at  representative 
research  site  or  operational 
site  for  relevant  time  period. 
Performance  requirements 
satisfied  and  documented. 

Validated  and  accepted 
by  stakeholders,  including 
budget  for  procurement 
and  ongoing  O&S. 

All  required  technical  approvals  have 
been  received.  Any  required  updates  to 
Unified  Facilities  Criteria  or  Guide  Speci¬ 
fications  have  been  made  or  in  process 
of  being  updated. 

6 

Adopted 

8 

In  operational  use  at  mul¬ 
tiple  installations. 

Training  and  communica¬ 
tion  programs  in  place. 

Technology  installed  and  in  operational 
use. 

Technology 

Typically,  TTPs  support  S&T  demonstrations  of  relatively  ma¬ 
ture  technologies— at  Technology  Readiness  Levels  (TRLs)  of 
5  and  above,  and  advance  to  a  TRL  7  or  8.  Often  a  candidate 
technology  will  be  demonstrated  in  a  research  environment— 
and  sometimes  in  an  operational  environment  on  a  Navy  in¬ 
stallation.  While  S&T  personnel  generally  well  understand  the 
technology  domain,  they  may  focus  primarily  on  the  readiness 
of  the  component  technology  under  study.  The  technology 
domain  encompasses  not  just  the  readiness  of  the  technology 
itself  but  also  its  integration  with  other  technologies,  including 
equipment  and  software.  Technology  that  performs  well  at  a 
component  level  may  not  be  suitable  for  integration  into  the 
installation  ecosystem. 

Operation  and  support  (O&S)  is  a  commonly  overlooked 
element  of  technology  integration.  Installation  maintenance 
personnel  typically  are  responsible  for  many  different  types 
of  equipment,  often  from  different  manufacturers  and  dif¬ 
ferent  vintages— e.g.,  one  building  with  a  brand-new  heating 
ventilation  and  air  conditioning  (HVAC)  system,  and  another 
with  a  20-year-old  HVAC  system.  If  a  new  technology  is  ad¬ 
opted,  it  does  not  necessarily  replace  every  instance  of  an  old 
technology,  and  the  installation  may  need  the  capability  to 
maintain  both  new  and  legacy  equipment  for  an  extended  pe¬ 
riod,  requiring  distinct  expertise,  spare  parts,  tools  and  other 
resources.  Technology  whose  use  or  maintenance  requires 


highly  specialized  training  may  not  be  adoptable  for  that  rea¬ 
son  alone.  Alternatively,  the  technology  may  be  adoptable  only 
with  additional  budgeting— both  funding  and  time— for  the 
training,  or  contracting  for  specialized  maintenance.  The  ARLs 
describe  the  need  for  identifying  and  documenting  any  training 
needed  during  the  demonstration  project. 

Stakeholders 

Stakeholders  are  individuals  or  entities  that  have  an  interest 
in  the  adoption  of  a  technology  or  the  ability  to  influence  its 
success.  A  common  pitfall  in  demonstration  projects  is  to  wait 
too  long  to  engage  all  relevant  stakeholders,  such  as  facilities 
engineers,  technical  approval  authorities  and  maintenance 
technicians.  The  ARLs  provide  a  framework  for  identifying  and 
engaging  stakeholders,  as  well  as  documenting  and  meeting 
their  needs. 

One  of  the  key  functions  of  TTPs  is  demonstration:  when  po¬ 
tential  end-users  can  see  a  technology  in  operation,  they  are 
much  more  likely  to  champion  its  adoption  into  their  organiza¬ 
tion.  As  highlighted  in  ARL  6,  TTPs  encourage  the  S&T  teams 
and  motivate  users  to  communicate  the  results  in  forums  such 
as  the  Federal  Energy  Exchange. 

Everywhere  in  DoD,  projects  must  compete  for  resources.  On 
the  installation  side,  the  trade-offs  often  are  made  at  the  instal¬ 
lation  level,  and  in  many  years  projects  compete  for  resources 
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with  high-profile  priorities.  Technologies  do  need  to  compete 
on  financials,  but  they  also  need  to  compete  for  end-user 
priority.  For  example,  energy  efficient  lighting  must  compete 
for  restoration  and  modernization  funds  head-to-head  with 
projects  like  hangar  upgrades,  pier  maintenance,  housing  and 
child  development  centers.  In  an  installation  context,  TTPs 
often  are  focused  on  a  particular  strategic  initiative,  such  as 
meeting  federal  installation  energy  mandates.  For  these  TTPs, 
there  may  be  short-  or  long-term  funding  sources  that  can 
be  targeted  to  support  the  technology.  For  large  projects,  the 
technology  may  be  adoptable  using  an  Energy  Savings  Per¬ 
formance  Contract.  S&T  engineers  should  identify  a  credible 
path  to  funding. 

A  further  challenge  for  installation  technologies  is  that  the  fi¬ 
nancial  analysis  may  differ  substantially  across  installations. 
For  example,  utility  costs  differ  across  installations;  not  only 
do  rates  per  kilowatt-hour  vary  widely,  but  some  utility  con¬ 
tracts  include  charges  for  high  peak  demands,  or  power  factor 
charges,  while  others  do  not.  Therefore,  it  is  not  uncommon  to 
see  an  incorrect  preliminary  financial  analysis  by  a  consultant 
or  an  analyst  unfamiliar  with  the  details  of  the  specific  utility 
contracts.  This  can  require  managers  to  modify  the  project 
before  it  is  able  to  compete  successfully,  delaying  or  even  pre¬ 
venting  adoption. 

Moreover,  utility  savings  often  are  realized  by  a  different  orga¬ 
nization  than  the  one  funding  the  investment,  and  the  financial 
case  may  need  to  include  allocating  future  savings  from  one 
budget  to  another.  For  some  technologies,  such  as  renew¬ 
able  energy,  there  is  an  ongoing  O&S  requirement.  Often  the 
decision  authority  for  sustainment  funding  is  different  from 
the  sponsor  for  the  initial  investment.  Even  for  a  demonstra¬ 
tion  project,  if  it  is  to  operate  long  term,  both  types  of  funding 
must  be  available.  For  a  successful  operational  adoption,  the 
resource  manager  for  sustainment  is  a  critical  stakeholder. 
This  highlights  the  importance  of  early  inclusion  of  all  relevant 
stakeholders  so  that  the  buy-in  from  those  responsible  for 
each  budget  is  supportive.  It  also  highlights  the  importance 
of  demonstrating  a  technology  in  a  DoD  installation,  so  that 
energy  managers  or  other  champions  can  point  to  a  success 
elsewhere  and  find  a  counterpart  of  each  needed  stakeholder 
at  the  demonstration  site. 

Processes 

The  processes  domain  includes  all  planning  and  budgeting 
processes  required  to  procure  a  technology,  as  well  as  any 
technical  approvals  required  before  a  technology  may  be 
used  at  an  installation.  There  is  some  overlap  between  the 
stakeholder  and  processes  domains,  as  some  of  the  hurdles  in 
the  processes  domain  create  stakeholders,  such  as  technical 
authorities.  The  stakeholder  domain  focuses  on  stakeholders 
who  have  an  ongoing  and  operational  interest  in  the  technol¬ 
ogy  and  who  provide  funding,  while  the  alignment  of  stake¬ 
holders  who  are  part  of  approval  and  authorization  processes 
and  whose  involvement  is  not  necessarily  ongoing  after  the 
technology  is  adopted  are  addressed  in  the  processes  domain. 


Even  if  funding  is  budgeted,  contracting  for  procurement 
can  still  present  a  hurdle  for  operational  adoption.  Fed¬ 
eral  acquisition  regulations  may  cause  delays  and  prevent 
adoption.  For  example,  if  specifications  for  a  demonstrated 
technology  are  too  narrow  and  there  are  only  one  or  two 
vendors,  it  may  be  difficult  to  contract.  S&T  engineers  may 
be  unaware  of  this  pitfall,  as  they  may  be  able  to  acquire 
technology  for  research  purposes  without  the  same  level 
of  competition  and  scrutiny. 

Safety,  environmental,  siting,  UFC  and  cybersecurity  require¬ 
ments  are  all  deal-breakers  if  they  are  not  addressed.  For  ex¬ 
ample,  the  UFC  currently  prohibit  the  use  of  stationary  lithium- 
ion  battery  systems  inside  occupied  structures.  A  technology 
using  lithium-ion  batteries  must  get  approval  or  a  waiver  for 
testing,  evaluation  and  validation.  If  the  technology  is  proven 
successful,  then  a  request  to  have  the  UFC  modified  would 
need  to  be  submitted  for  installation  and  use  at  other  facilities. 
It  is  very  important  for  S&T  engineers  to  identify  the  technical 
authority  and  work  with  them  early  to  identify  the  require¬ 
ments  and  make  any  adjustments  necessary  to  the  demon¬ 
stration  project,  and  anticipate  how  a  facility  manager  would 
handle  the  same  requirements. 

Cyber  and  information  assurance  authorities  should  be  in¬ 
volved  as  soon  as  possible— it  is  generally  much  more  difficult 
to  fix  issues  later  than  early  in  the  development  process.  If  a 
demonstration  project  requires  a  special  waiver  that  would 
not  be  readily  available  in  a  larger  or  more  permanent  adop¬ 
tion,  the  S&T  team  should  work  to  identify  a  path  to  meet¬ 
ing  the  requirements  in  a  wider  adoption.  Some  technologies 
may  require  changes  to  the  UFC,  and  this  requires  working 
with  the  criteria  managers  early  on.  S&T  engineers  should 
also  consider  whether  scale  would  change  the  ability  of  the 
technology  to  get  approvals— would  a  larger  project  trigger  a 
different  standard  or  level  of  scrutiny?  As  described  for  ARLs 
4  and  5,  S&T  teams  should  document  processes,  as  well  as 
approvals,  for  reference  by  future  managers  who  may  wish  to 
adopt  the  technology. 

TTPs  can  greatly  facilitate  the  adoption  of  valuable  technolo¬ 
gies.  Identification  of  process  hurdles  and  involvement  of 
stakeholders  while  a  technology  is  under  development  as  part 
of  a  TTP  is  very  important  for  transition.  When  stakeholder 
and  process  alignment  are  considered  early  in  each  project, 
technology  transfer  in  TTPs  becomes  faster,  cheaper  and  less 
risky.  Failing  to  anticipate  stakeholder  and  process  hurdles 
often  leads  to  situations  in  which  technology  cannot  be  ad¬ 
opted  because  critical  elements  were  not  addressed  during  the 
demonstration/validation  phase,  such  as  technical  approval  or 
identification  of  a  transition  resource  sponsor  and  vehicle.  The 
ARLs  provide  a  general  framework  for  anticipating  hurdles  and 
developing  milestones  for  demonstration  projects  in  TTPs.  & 


The  authors  can  be  contacted  at  eregnier@nps.edu,  rbarron@ku.edu, 
danussba@nps.edu,  and  kail.macias@navy.mil. 
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The  "valley  of  death"  between  technology  development  efforts  and  produc¬ 
tion  programs  has  long  been  a  problem  in  the  government  and  private  in¬ 
dustry.  Despite  the  U.S.  Special  Operations  Command's  (SOCOM)  reputa¬ 
tion  for  agile  development  and  rapid  acquisition,  the  same  has  been  true  for 
SOCOM.  This  article  focuses  on  the  development  of  a  new  methodology 
to  capture  discrete  actions  in  preparation  for  a  technology  transition  and  measure 
organizational  confidence  in  the  success  of  that  transition.  Initial  indications  are  that 
this  process  significantly  increases  the  likelihood  of  successful  technology  transi¬ 
tion  and  that  the  associated  metrics  and  methodology  could  be  quickly  and  easily 
adopted  by  other  acquisition  organizations  to  help  them  bridge  their  own  "valleys 
of  death"  and  avoid  failed  or  suboptimal  transitions. 


Davis  is  the  U.S.  Special  Operations  Command  (USSOCOM)  director  of  agile  acquisition.  He  previously  was  the  USSOCOM  program  ex¬ 
ecutive  officer  for  command ,  control ,  communications and  computer  (C4)  systems  and  director  of  science  and  technology.  Ballenger  is  an 
aviation  systems  analyst  with  JHNA,  Inc.  A  retired  U.S.  Army  officer,  he  provides  contract  science  and  technology  support  to  USSOCOM. 
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In  2014,  the  command's  Special  Operations  Forces  Acqui¬ 
sition,  Technology,  and  Logistics  organization  (SOF  AT&L) 
began  trying  to  address  its  transition  shortcomings  by  moving 
an  experienced,  proven  SOF  program  executive  officer  (PEO) 
to  direct  the  Science  and  Technology  (S&T)  organization.  The 
PEO  previously  was  quite  vocal  regarding  the  command's  lack 
of  success  in  regularly  transitioning  technologies  to  a  program 
of  record.  After  roughly  a  year  in  the  S&T  position,  numerous 
changes  had  been  made  to  increase  the  likelihood  of  success¬ 
ful  transitions.  Despite  those  efforts,  the  S&T  director  still  had 
no  real  way  to  measure  or  predict  the  probability  of  transition 
success  either  for  individual  projects  or  across  the  portfolio. 
A  team  was  chartered  to  look  at  appropriate  leading  and  fol¬ 
lowing  metrics  and  began  work  on  the  problem. 

During  the  research  process,  the  team  identified  a  separate 
but  related  issue.  While  the  S&J  project  managers  had  a  clear 
understanding  that  transition  of  their  technology  was  a  desired 
outcome,  there  was  little  common  ground  between  that  and 
the  mandate  of  the  PEOs'  program  managers  who  were  driven 
by  cost,  schedule  and  the  performance  of  their  existing  pro¬ 
grammatic  acquisition  strategy. 

So,  the  final  challenge  to  the  team  was  to  (1)  develop  a  series  of 
metrics  to  measure  the  transition  success  of  each  S&T  project, 
(2)  ensure  those  metrics  could  be  aggregated  to  the  portfo¬ 
lio  level,  and  (3)  incorporate  a  mechanism  that  ensured  S&T 
project  managers  and  PEO  program  managers  would  have  a 
common  understanding  of  the  mechanisms  and  motivations 
for  transition. 

The  search  for  appropriate  tools  began  with  some  known  con¬ 
straints.  Ideally,  a  transition  support  metric  would  be  easy  to 
implement  and  actually  decrease  workload  for  portfolio  man¬ 


agement.  It  must  fit  within  funding  realities  and  existing  data 
infrastructure.  It  must  reflect  the  important  balance  between 
innovation  opportunities  and  operational  outcomes.  To  mini¬ 
mize  cultural  resistance  to  adoption,  it  must  avoid  external 
benchmarking  as  measures  of  success.  Most  importantly,  it 
must  support  the  SOCOM  SOF  AT&L  customer. 

Open-source  research  revealed  a  common  theme  across 
government  and  commercial  development.  While  the  in¬ 
gredients  and  pathways  of  technological  progress  are  well 
understood,  there  are  few  best-practice  or  standard  mecha¬ 
nisms  to  measure  and  manage  technology  transition  efforts. 
In  some  cases,  projects  were  initiated  or  even  completed 
before  transition  potential  was  determined.  In  other  cases, 
project  initiation  required  approval  from  an  external  over¬ 
sight  council  to  ensure  alignment  with  the  program  enter¬ 
prise.  Neither  of  these  extreme  approaches  are  appropriate 
for  SOCOM  S&T  implementation.  The  search  continued  for 
a  solution  between  these  extremes. 

The  Government  Accountability  Office  (GAO)  has  studied 
this  issue  for  more  than  40  years.  In  multiple  reports  dating 
back  to  1974,  GAO  has  called  for  better  transition  metrics  and 
more  active  management  of  transition  efforts.  In  recent  years, 
they  highlighted  the  success  of  transition  commitment  metrics 
used  by  the  Joint  Capability  Technology  Demonstration  and 
Future  Naval  Capabilities  programs.  These  scales  scored  each 
project  by  whether  a  transition  agreement  was  complete,  in 
progress  or  absent.  Implementation  of  standardized  transition 
assessment  was  a  step  in  the  right  direction. 

The  innovation  environment  at  SOCOM  AT&L  encourages  risk 
taking  in  S&T.  Signed  transition  agreements  represent  a  very 
high  standard  for  projects.  Special  Operations  PEOs  seek  to 
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retain  their  programs'  agility  and  will  not  readily  commit  to 
unproven  solutions.  A  transition  commitment  metric  tailored 
for  use  in  SOCOM  S&T  needs  to  recognize  more  incremental 
precursor  steps.  The  Technology  Readiness  Level  (TRL)  scale 
fills  a  similar  role  in  the  realm  of  technology  risk.  GAO  recom¬ 
mended  DoD-wide  adoption  of  TRL  in  1999  following  success¬ 
ful  use  by  NASA  and  the  U.S.  Air  Force.  It  is  well-understood, 
universally  accepted,  and  applicable  across  a  wide  variety  of 
technologies.  It  is  as  useful  as  it  is  simple.  We  set  out  to  es¬ 
tablish  a  similar  tool  for  transition  management. 

The  simplicity  and  applicability  of  TRL  became  the  tailoring 
benchmark  for  a  new  transition  commitment  metric.  The  team 
first  replaced  the  term  commitment  with  confidence  to  bet¬ 
ter  reflect  a  dynamic  continuum  rather  than  a  binary  condi¬ 
tion.  The  new  Transition  Confidence  Level  (TCL)  scale  has  the 
same  numerical  range  and  objective  accomplishment-based 
approach  as  the  TRL  scale.  The  1-9  scaling  was  initiated  as 
a  matter  of  convenience  but  later  proved  to  support  some 
compelling  data  visualization  relative  to  TRL.  The  steps  fol¬ 
low  a  logical  arc  from  uncertainty  to  a  completed  transition, 
as  shown  in  Table  1. 

Like  the  TRL  chart,  the  steps  enable  status  scoring  for  a  proj¬ 
ect,  and  they  form  a  roadmap  for  progress  and  coordination 
typically  needed  for  transition  success.  In  that  sense,  the  TCL 
chart  is  both  a  scorecard  and  a  checklist.  The  defining  char¬ 
acteristics  of  each  level  are  tailorable  to  organizational  behav¬ 
iors  or  changing  dynamics  between  technology  developers 
and  PEO  leaders.  The  chart  retains  its  usefulness  as  long  as  it 
represents  the  organization's  desired  steps  between  project 
initiation  inputs  and  completed  transitions.  The  current  itera¬ 
tion  allows  a  project  to  proceed  to  TCL  4  dependent  only  on 
internal  S&T  Directorate  activities.  These  precursor  steps  pro¬ 
vide  a  progress  report  on  the  S&T  team's  transition  planning 
during  initial  project  incubation.  Advancement  to  TCL  5  and 
beyond  requires  explicit  cooperation  and  increasing  coordina¬ 
tion  with  a  program  office.  A  project  at  TCL  7  and  8  merits 
senior  leader  attention  to  ensure  high-level  coordination  for 
funding,  contract  actions  and  organizational  handover.  We 
expect  the  contents  of  the  chart  to  evolve  to  meet  emerg¬ 
ing  process  changes  and  support  maturing  relationships  with 
transition  stakeholders. 

Implementation  of  the  TCL  metric  included  workforce  training, 
project  assessments,  TCL  chart  configuration  management, 
and  incorporation  of  TCL  data  entry  into  the  Directorate's 
knowledge  management  portal.  Workforce  training  was  not 
difficult.  Each  technologist  and  project  manager  was  already 
familiar  with  transition  planning,  command  expectations, 
and  the  use  of  similar  tools  like  the  TRL  scale.  Introduction 
of  TCL  simply  assigned  a  number  and  standardized  a  report¬ 
ing  framework  for  a  process  the  workforce  members  already 
were  executing.  Project  assessments  were  straightforward. 
The  technology  transition  lead  for  the  Directorate  became  the 
configuration  manager  for  the  TCL  chart  and  would  control  its 
contents  and  evolution.  The  knowledge  management  portal 


Table  1.  Transition  Confidence  Level  Scale 


Level 

Characteristics 

9 

•  Transition  to  PEO  funding  and  management  com¬ 
pleted 

■  Transition  After  Action  Report  and  storyboard 
documented  on  S&T  portal 

•  Transition  success  report  to  AT&L 

8 

•  Signed  transition  agreement  between  PM  and  S&T 

•  Transition  funding  committed 

7 

•  Integration  strategy  defined 

•  Transition  cost  estimate  complete 

■  Potential  funding  sources  identified 

6 

•  Transition  technical  goals  approved  by  PM,  S&T 
■  Transition  schedule  estimate  developed 

•  Project  included  in  PM  plans  as  a  potential  source 

5 

■  Expressed  interest  from  PM  office 
•  Active  communication  with  named  PM  contact 

4 

■  Target  PMs  briefed  and  provided  progress  updates 

•  Key  transition  stakeholders  named 

•  Relevant  programs  named 

3 

•  Specific  project  technical  goals  established 

•  Target  acquisition  programs  identified 

■  Potential  transition  stakeholders  identified 

2 

•  Project  initiated 
■  TRL  goals  established  (baseline) 

1 

•  Working  Group  interest  expressed 
■  Active  tech  discovery 

•  Acknowledged  gap 

Figures  and  tables  by  the  authors. 


modification  was  completed  via  established  change  request 
procedures.  Of  note,  the  data  entry  method  for  the  portal  did 
not  include  TCL  definitions,  only  the  number.  This  decoupled 
configuration  management  of  the  TCL  scale  from  the  portal 
modification  process.  Once  each  project  had  a  TCL  value  and 
action  officers  could  keep  that  value  updated  in  the  portal, 
management  metrics  can  be  extracted  to  inform  portfolio 
decisions  across  diverse  efforts  and  projects. 

The  implemented  TCL  metric  enables  consistent,  uniform 
discussions  of  transition  likelihood  across  different  types  of 
technologies.  The  steps  capture  the  organization's  pathway 
for  S&T  and  program  coordination,  encouraging  both  sides  of 
the  "valley  of  death"  to  lean  toward  each  other  to  close  the  gap. 
Especially  for  those  steps  requiring  accord  between  S&T  lead¬ 
ers  and  program  managers,  it  provides  a  dispassionate,  objec¬ 
tive  framework  for  discussions  and  organizational  progress. 
It  makes  project  relevance  and  transition  outcomes  a  part  of 
every  project  discussion  while  contributing  to  portfolio  trans¬ 
parency.  The  ability  to  adapt  the  characteristics  of  each  level 
ensures  relevance  as  organizational  relationships  and  needs 
change.  Finally,  TCL  can  quickly  cue  leaders  in  both  the  S&T 
and  program  spheres  to  imbalances  in  the  portfolio.  The  ability 
to  quickly  identify  outliers  allows  leaders  to  allocate  their  time 
and  attention  where  they  are  needed  most. 
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Figure  1.  Portfolio  Average  TRL  and  TCL 


Average  TRL 


Average  TCL 


Current 


60  days  ago  (or  last  report) 

1 20  days  ago  (or  two  reports  ago) 


At  the  individual  proj¬ 
ect  level,  TCL  quantifies 
a  project's  transition 
status.  At  the  portfo¬ 
lio  level,  it  provides  an 
organizational  health 
indicator  that  can 
cue  leader  decisions. 

While  individual  proj¬ 
ect  officers  strive  for 
the  highest  TCL  pos¬ 
sible  for  their  projects, 
a  very  high  average  TCL 
for  the  entire  portfolio 
may  indicate  inappro¬ 
priate  risk  avoidance.  If 
every  project  will  tran¬ 
sition,  the  valiant  failures  of  a  dynamic  research  organization 
are  missing.  Conversely,  a  very  low  average  TCL  may  indi¬ 
cate  a  lack  of  relevance  to  supported  programs.  In  the  case 
of  SOCOM  S&T,  the  target  TCL  is  intended  to  hover  between 
4  and  7.  It  will  probably  reflect  some  seasonality  under  fiscal 
rules  as  cohorts  of  new  projects  will  drive  down  portfolio  TCL 
upon  initiation.  As  projects  mature,  the  TCL  will  increase  until 
driven  down  by  a  new  class  of  projects  with  the  following  year's 
appropriation.  Likewise,  once  projects  complete  their  transi¬ 
tion  and  leave  the  portfolio,  their  high  TCL  scores  are  removed 
from  the  equation  to  be  replaced  by  lower  TCL  new  projects. 
While  not  directly  coupled,  average  TRL  of  the  portfolio  will 
follow  similar  ebbs  and  flows.  An  example  visualization  of  av¬ 
erage  TRL  and  TCL  is  shown  in  Figure  1. 

The  ability  to  measure  transition  confidence  in  a  scale  cali¬ 
brated  to  technology  readiness  enables  some  helpful  visu¬ 


l - Directorate  Goals - - 

- AT&L  Guidance 


alization.  The  hypothetical  S&T  portfolio  in  Table  2  includes 
data  for  current  TRL,  current  TCL,  and  budget.  A  quick  graphic 
presents  a  powerful  visual  tool,  shown  in  Figure  2.  Money  and 
time  will  tend  to  move  projects  to  the  right.  Project  relevance 
and  program  office  coordination  will  tend  to  move  projects  to¬ 
ward  the  top.  Relative  budget  size  is  an  indicator  of  command 
priority  and  risk  tolerance.  Taken  together,  these  metrics  reveal 
that  expensive  projects  in  the  bottom  right  of  the  chart  might 
be  consuming  resources  best  spent  on  projects  at  the  top  left 
of  the  chart.  No  specific  behavior  rules  are  needed.  The  chart 
is  a  decision-support  tool  that  graphically  presents  key  data  for 
numerous  projects  to  enable  leaders  to  make  more  informed 
decisions  no  matter  the  trade  space. 

Because  TCL  does  not  invoke  any  external  standards,  S&T 
organizations  are  only  making  internal  comparisons.  This  al¬ 
leviates  concerns  about  different  missions,  stakeholders  and 


Figure  2 .  Hypothetical  TRL ,  TCL,  and  Budget  Data  Visualization 
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desired  outcomes  amongst  the  many  diverse  devel¬ 
opment  organizations.  Leaders  can  set  their  own 
internal  goals  and  manage  against  them. 

TCL  can  also  contribute  to  project  storyboards  for 
both  current  status  and  archiving.  When  combined 
with  TRL  and  financial  execution  data  and  goals  over 
time,  a  powerful  visualization  is  formed  showing  a 
single  timeline  of  obligations,  expenditures,  TRL, 
and  TCL;  an  example  is  shown  in  Figure  3.  Using 
averages  for  TRL  and  TCL,  the  storyboard  can  cover 
multiple  projects  within  a  function  or  the  entire 
portfolio  to  compare  performance  between  divi¬ 
sions  or  year  to  year. 

SOCOM  S&T  has  implemented  TCL,  and  requires 
its  project  managers  to  track  and  report  the  mea¬ 
sure  along  with  TRL  for  each  of  their  projects  on  a 
recurring  basis.  The  lack  of  subjectivity  in  the  scale 
makes  it  easy  to  score  projects,  monitor  progress 
over  time,  and  quickly  assess  average  TCL  for  the 
entire  portfolio  or  other  subordinate  areas.  TCL 
quickly  identifies  the  outliers,  allowing  leadership 
to  concentrate  on  candidates  for  more  direct  senior 
coordination,  candidates  for  divestment,  and  candi¬ 
dates  requiring  additional  funding  versus  projects 
"on  glideslope"  for  transition.  The  data  and  visual¬ 
izations  can  be  used  explicitly  for  a  management 
by  exception  approach  or  as  a  tailorable  decision 
support  tool  for  portfolio  management. 


Table  2.  Hypothetical  S&T  Project  Data 


Project  Name 

Budgeted 

TRL  Now 

Current  TCL 

Digital  Data  Infused  Optics 

$250,000 

8 

4 

Man  Portable  AntiTank  Wpn 

$340,000 

6 

4 

Maritime  Waveform 

$625,000 

5 

5 

Transparent  Cranial  Armor 

$280,000 

6 

3 

Anti  Personnel  Munition 

$650,000 

7 

3 

Rapid  Diagnosis  Kit 

$325,000 

6 

4 

Laser  Source  Geolocation 

$295,000 

3 

4 

Personal  Aerial  Vehicle 

$200,000 

2 

6 

Modular  Exoskeleton 

$450,000 

9 

7 

Through  Wall  Sensor 

$325,000 

6 

5 

Expendable  ISR 

$380,000 

5 

4 

DE  Slewable  Mirror 

$180,000 

3 

7 

Diver  Thermal  Control 

$225,000 

6 

7 

Canine  Triage  Kit 

$215,000 

6 

6 

RPG  Defeat 

$545,000 

5 

4 

Secure  Squad  Wireless  4G 

$350,000 

5 

5 

Autonomous  Vehicle  Kit 

$345,000 

4 

3 

UAS  Precision  Drop 

$290,000 

5 

4 

Sniper  Airburst  Round 

$430,000 

6 

7 

Squad  Data  Gateway 

$360,000 

4 

4 

Nutraceutical  Study 

$150,000 

4 

3 

The  adoption  of  TCL  has  provided  a  wealth  of  insight 
into  the  progress  of  the  S&T  portfolio  toward  tran¬ 
sition  with  a  minimum  of  additional  data  entry.  Additionally, 
the  presence  of  this  data  on  SOF  AT&L's  real  time  dashboard 
provides  complete  transparency  and  understanding  between 
the  project  manager,  S&T  director,  program  manager  and  PEO. 
The  command  believes  the  tool  has  immediate  potential  ap¬ 
plication  to  numerous 
S&J  organizations  and 
portfolios  and  is  easily 
adaptable  to  fit  each 
organization's  particu¬ 
lar  needs. 


how  to  improve  the  metrics  or  visualizations,  and  is  interested 
in  discussing  those  ideas  further. 

The  authors  can  be  contacted  at  anthony.davis@socom.mil  and  tom.bal- 
lenger@jhna.com. 


Figure  3.  Hypothetical  Project  Storyboard 


SOCOM  S&T  plans  to 
continue  use  of  TCL 
and  TRL  as  comple¬ 
mentary  measures  of 
project  performance, 
and  will  continue 
maturing  visualiza¬ 
tion  tools  to  support 
informed  leadership 
decision  making.  The 
command  welcomes 
any  inputs  or  ideas  for 
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The  first  in  a  series  of  articles 


EVM  System's 
High  Cost 

Fact  or  Fiction? 

Ivan  Bembers 
Ed  Knox 
Michelle  Jones 
Jeff  Traczyk 


t  seems  like  every  year  the  writing  of  budgets  sparks  proposals  to  eliminate  the  Earned  Value 
Management  System  (EVMS)  requirements  on  federal  acquisitions  as  a  way  to  save  millions 
of  dollars.  People  suggest  the  government  can  do  away  with  EVMS  in  favor  of  more  efficient 
and  affordable  management  techniques.  But  is  there  a  basis  for  these  assertions?  For  more 
than  50  years,  the  Department  of  Defense  (DoD)  has  recognized  the  power  of  both  EVMS 
and  the  Cost/Schedule  Control  Systems  Criteria  (C/SCSC),  the  forerunner  to  EVMS,  and  has 
kept  EVMS  requirements  in  place  to  promote  sound  planning  and  effective  program  execution. 


The  Government  Accountability  Office  (GAO)  has  consistently  cited  EVMS  as  providing  a  comprehensive  early 
warning  of  potential  cost  and  schedule  overruns.  Furthermore,  most  major  aerospace  and  defense  (A&D)  indus¬ 
try  partners  have  made  the  business  decision  to  adopt  EVMS  as  a  standard  way  of  doing  business  on  all  types  of 


Bembers  is  the  chief  of  the  National  Reconnaissance  Office  (NRO)  Earned  Value  Management  (EVM)  Center  of  Excellence  (ECE).  He  is 
responsible  for  EVM  system  acceptance  and  surveillance  reviews ,  facilitating  integrated  baseline  reviews  and  supporting  programs'  use  of 
EVM  across  the  enterprise.  Knox,  Jones  and  Traczyk  support  EVM  at  the  NRO. 
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Contractor  Mandated 
EVMS  Requirements  for 
ALL  Commercial  or  Fixed 
Price  Contracts 


Contractor  A 


Contractor  B 


Contractor  C 
Contractor  D 


development  work,  includ¬ 
ing  commercial,  fixed-price, 
and  government  cost  type 
contracts.  Why  then  do  we 
hear  the  yearly  grumbling 
about  the  burden  of  EVMS 
on  government  programs? 

In  1994,  Coopers  &  Lybrand/ 

TASC  (CLT)  performed  a 
study  that  included  an  as¬ 
sessment  of  the  cost  of  C/ 

SCSC.  The  CLT  study,  often 
cited  as  the  definitive  source 
in  this  matter,  concluded  that 
there  was  a  0.9  percent  DoD 
regulatory  cost  premium  for 
C/SCSC  on  government 
contracts.  (The  study  noted, 
however,  that  the  majority  of 
the  cost  premium  resulted 
from  excessive  requirements 
that  were  not  inherent  in  C/ 

SCSC.)  Following  that  ef¬ 
fort,  Dr.  David  S.  Christensen 
consolidated  a  number  of 

other  studies  in  The  Costs  and  Benefits  of  the  Earned  Value 
Management  Process  and  identified  the  cost  of  EVMS  to  be 
somewhere  between  0.1  percent  to  5  percent  of  the  contract 
value.  While  these  studies  provide  excellent  information,  all  of 
their  supporting  data  was  gathered  before  1996,  when  indus¬ 
try  took  more  ownership  of  EVMS.  In  December  of  that  year, 
Under  Secretary  of  Defense  (Acquisition  and  Technology)  Dr. 
Paul  Kaminski  accepted  industry's  32  guidelines  for  EVMS 
and  rolled  them  into 
the  1997  DoD  In- 


Figure  1.  NRO  Study  of  EVMS  Requirements 


Contractor  E 


National  Reconnaissance  Office 
EVMS  Requirements 


<  $50M 


EVMS  NOT  REQUIRED 
♦  IMS  Required 


CD 

1  i£ 

/  >=  $50M  &\ 

+  EVMS  REQUIRED 

£  =5 

~  TO 

\  <$100M 

♦  CPR/IPMR  Required 

^  > 

(S)  TZ 

♦  IMS  Required 

CD 

Q  O 

<  -o 

oc  gj 

<  o 

EVMS  REQUIRED 

S2 

<K  >=$100M 

^  ♦  CPR/IPMR  Required 

o 

-t->  to 

U 

♦  IMS  Required 

TO 

£  > 

♦  Surveillance  Required 

C  LU 

o 

u 

and  discovered  that  every  prime  contractor  reviewed  had  an 
internal  EVMS  threshold  for  in-house,  commercial  or  fixed- 
price  efforts  that  was  much  lower  than  the  requirement  for 
NRO  acquisitions  (Figure  1). 

If  major  industry  partners  within  A&D  rely  on  and  use  EVMS 
for  government  cost  type,  fixed-price  and  commercial  efforts, 
and  they  have  a  management  control  system  in  place,  then 


struction  5000. 2R. 
By  July  1998,  the 
guidelines  were  for¬ 
mally  issued  as  an 
American  National 
Standards  Institute/ 
Electronic  Industries 
Alliance  document, 
creating  a  national 
EVMS  standard 
that  was  applicable 
beyond  DoD  pro¬ 
grams.  To  this  point, 
in  2014,  the  National 
Reconnaissance  Of¬ 
fice  (NRO)  Earned 
Value  Management 
Center  of  Excellence 
(ECE)  examined  a 
large  number  of  its 
major  acquisitions 


Figure  2.  The  Joint  Space  Cost  Council  Better  EVMS 
Implementation  Study 


CD 

i/i 

.c 

CL 


JCSS  Industry 
Day 

(joint  Govern¬ 
ment/Industry 
participation) 


Identification 

of  78  Industry 
Cost  Areas 


Industry  Survey 

to  assess  cost 
areas  as  high, 
medium,  low,  no 
impact 


Phase  I 

Recommendation 

Report, 

focusing  on  high 
and  medium  cost 
impact  areas 


JCSS  Government 
Day 

(joint  Govern¬ 
ment/Industry 
participation) 


Identification 

of  EVMS  Products 
and  Management 
Activities  used  by 
the  Government 


Government 

Survey 

assessed  areas 
based  on  Value 


Phase  II 

Recommendation 

Report, 

focusing  on  PM 
value  assessment 
areas 


Joint 

Government/ 
Industry 
Implementa¬ 
tion  Plan 


19 


Defense  AT&L:  January-February  2017 


EARNED  VALUE  MANAGEMENT 


Figure  3.  Phase  I  Survey  Results 
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DCMA=Defense  Contract  Management  Agency;  KTR=contractor;  PARCA=DoD  Office  of  Perfor¬ 
mance  Assessment  and  Root  Cause  Analyses 


several  questions  need  to  be  answered  regarding  any  real  or 
perceived  additional  costs  of  implementing  EVMS  on  govern¬ 
ment  programs: 

■  What  are  the  differences  in  applying  EVMS  on  a  govern¬ 
ment  cost  type  contract  versus  a  commercial  or  fixed-price 
contract  (including  reporting  requirements)? 

■  What  are  the  underlying  costs  of  these  differences? 

■  What  is  the  government  value  and  derived  benefit  of  these 
differences  (i.e.,  additional  deliverables  or  requirements  that 
substantially  help  manage  the  program)? 

■  Are  there  opportunities  to  generate  better  efficiencies  for 
these  differences,  especially  in  areas  with  claims  of  high  cost 
and  low  value? 

In  2013,  the  Joint  Space  Cost  Council  (JSCC)  initiated  the 
Better  EVMS  Implementation  Study  to  address  these  questions. 
The  JSCC  was  established  by  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics  and  the  Under  Sec¬ 
retary  of  the  Air  Force  as  a  joint  government  and  industry 
forum  with  a  commitment  to  affordable,  accurate  and  cred¬ 
ible  cost  estimating  on  space  systems.  The  JSCC  also  actively 
addresses  cost  estimating  and  earned  value  management 
issues  with  the  goal  of  improving  cost-estimating  accuracy 
and  the  betterment  of  earned  value  management  practices, 
which  in  turn  affect  budget  realism  and  improve  schedule 
and  program  execution. 

Due  to  the  study's  extensive  scope,  the  JSCC  divided  it  into 
two  parts  (Figure  2).  Phase  I  (completed  in  2015)  targeted 
industry  and  was  designed  to  understand  the  delta  implemen¬ 
tation  cost  impacts  of  EVMS  required  for  a  government  cost 


type  contract  versus  EVMS 
performed  on  a  commercial, 
internal,  or  fixed-price  ef¬ 
fort.  Phase  II  (completed  in 
2016)  targeted  the  federal 
government  program  man¬ 
agers  (PMs)  and  focused  on 
understanding  how  those 
PMs  value  and  use  EVMS 
products  and  management 
activities  (P&MA). 

During  Phase  I,  the  JSCC 
collected  information  from 
46  separate  space  pro¬ 
grams  ranging  in  value  from 
$20  million  to  more  than  $5 
billion  at  Ball  Aerospace, 
Boeing,  Northrop  Grum¬ 
man,  Lockheed  Martin  and 
Raytheon.  This  phase  used 
survey  responses  to  ana¬ 
lyze  78  specific  cost  areas 
identified  by  industry  as  the 
key  cost  drivers  (real  or  per¬ 
ceived)  for  applying  EVMS  on  government  cost  type  contracts. 
As  shown  in  Figure  3,  nearly  73  percent  of  survey  responses 
identified  the  cost  areas  as  No  or  Low  Impact  on  the  cost  of 
EVMS,  and  the  data  identified  government  program  manage¬ 
ment  as  the  primary  stakeholder  driving  High  and  Medium 
Impacts.  Furthermore,  cost  impacts  were  scattered  among 
all  78  cost  areas,  and  no  single  cost  area  was  identified  as  a 
High  or  Medium  Impact  across  a  majority  of  the  programs 
that  participated  in  the  study. 

Using  Phase  I  data,  the  JSCC  identified  three  overall  themes 
regardingthe  cost  of  EVMS.  First,  Control  Account  (CA)  level 
(size  and  number)  significantly  affects  the  cost  of  EVMS.  Sec¬ 
ond,  program  volatility  and  lack  of  clarity  about  the  program's 
scope  as  well  as  funding  uncertainty  may  affect  the  cost  of 
EVMS,  just  as  any  other  program  management  discipline. 
Third,  volume  of  reviews  (including  surveillance,  compliance, 
and  Integrated  Baseline  Reviews  [IBRs])  as  well  as  the  incon¬ 
sistent  interpretation  of  the  32  Guidelines  affects  the  cost  of 
EVMS  (this  theme  could  have  two  separate  parts,  but  was 
originally  based  on  industry's  interpretation  of  all  government 
reviews).  In  April  2015,  these  themes  were  published  in  Better 
EVMS  Implementation  Themes  and  Recommendations  along  with 
specific  recommendations  to  reduce  cost. 

Government  and  industry  EVMS  subject-matter  experts 
(SMEs)  also  made  several  other  observations  regarding 
Phase  I: 

■  Inconsistent  government  PM  application  of  EVMS  require¬ 
ments  appears  to  be  the  leading  driver  of  High  and  Medium 
Impacts  to  the  cost  of  EVMS. 
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■  Inconsistent  assessment  of  the 
materiality  of  Surveillance  Review 
findings  can  affect  the  cost  of 
EVMS  (e.g.,  reviewer's  experience 
level,  approach,  etc.). 

■  The  JSCC  survey  data  does  not 
substantiate  the  numerous  anec¬ 
dotal  perceptions  of  major  earned 
value-related  cost  impacts  (e.g., 
IBRs  cost  too  much,  etc.). 

During  Phase  II,  the  JSCC  interviewed 
32  government  PMs  at  Air  Force 
Space  and  Missile  Systems  Center, 
the  National  Aeronautics  and  Space 
Administration,  and  the  NRO  to  as¬ 
sess  the  government  value  of  EVMS. 
The  JSCC  assessed  12  specific  EVMS 
P&MA  ranging  from  Earned  Value 
Data  by  Work  Breakdown  Structure 
(WBS),  commonly  known  as  Con¬ 
tract  Performance  Report/Integrated 
Program  Management  Report  (CPR/ 
IPMR)  Format  1  to  Over  Target  Base¬ 
line/Schedule  (OTB/OTS).  As  shown 
in  Figure  4,  the  results  indicated  that 
Integrated  Master  Schedule  (IMS), 
the  IBR  and  EVM  Metrics  were  the 
most  highly  valued  P&MA  by  govern¬ 
ment  PMs.  Even  the  lowest  scoring 
P&MA,  such  as  Earned  Value  Data 
by  OBS  (CPR/IPMR  Format  2)  and 
Integrated  Master  Plan  (IMP),  were 
identified  as  having  medium  value. 


Figure  4.  Phase  II  Survey 
Results 
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In  September  2016,  the  JSCC  updated 

Better  EVMS  Implementation  Themes  and  Recommendations  to 
include  a  series  of  recommendations  on  ways  to  increase  value 
for  each  EVMS  P&MA  assessed  in  Phase  II.  These  recom¬ 
mendations  are  rooted  in  the  use  of 
best  management  practices  as  well 
as  education  on  how  to  use  these 
EVMS  P&MA  better.  As  with  Phase 
I,  government  and  industry  partici¬ 
pants  also  provided  additional  ob¬ 
servations: 


Most  government  PMs  have  a 
strong  appreciation  of  EVMS— in 
many  cases,  they  assess  P&MA 
at  the  highest  possible  level  and 
identify  a  heavy  reliance  of  EVMS 
metrics  during  program  execution. 
Most  government  PMs  recognize 
the  value  of  the  IBR  to  generate  a 
valid  and  executable  Performance 
Measurement  Baseline  (PMB). 


■  Some  government  PMs  do  not  fully 
recognize  how  surveillance  can  sup¬ 
port  their  need  to  provide  higher 
quality  data  (e.g.,  one  PM  contends 
that  since  he  is  continually  walking 
the  factory  floor,  he  does  not  learn 
anything  new  from  an  independent 
surveillance). 

Once  Phase  II  was  completed,  the 
JSCC  relied  on  government  and  indus¬ 
try  SMEs  to  integrate  the  analysis  of 
data  collected  during  both  phases  of 
the  study.  Using  a  matrix  of  the  Phase 
I  cost  areas  versus  the  Phase  II  EVMS 
P&MA,  the  SMEs  applied  the  premise, 
"The  Customer  requirement  for  EVMS 
Product/Management  Practice  X  can 
influence  Cost  Area  Y,"  to  determine 
the  direct  relationship  of  a  particu¬ 
lar  EVMS  P&MA  with  a  specific  cost 
impact.  While  the  assessments  were 
subjective,  the  SMEs  required  consen¬ 
sus  on  936  specific  matrix  intersections 
and  discussed  all  dissenting  opinions  to 
generate  the  best  possible  evaluation  for 
each  intersection. 

As  shown  in  Figure  5,  the  result  of  this 
synthesis  shows  that  each  EVMS  P&MA 
is  in  the  High-Value  and  Low-Cost  quad¬ 
rant.  Additionally,  analysis  of  Phase  I  and 
Phase  II  data  also  indicates  that  every 
EVMS  P&MA  except  Surveillance,  IBR, 
and  Data  by  WBS  shares  100  percent  of 
its  associated  cost  impacts  with  other 
P&MA  (Figure  6).  This  means  that  even  though  a  particular 
product  such  as  Data  by  OBS  may  have  a  lower  government 
value,  the  elimination  of  this  product  will  most  likely  have  a 


Figure  5.  Products  and  Management  Activities 
Cost  and  Value 
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EARNED  VALUE  MANAGEMENT 


Figure  6.  Shared  Versus  Non-Shared 
Cost  Impacts 
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limited  effect  on  reducing  the  cost  of  EVMS  since  its  associ¬ 
ated  cost  impacts  will  still  exist  on  other  P&MA. 

Although  the  JSCC  performed  its  study  solely  on  space-related 
programs,  the  contractors  who  participated  in  Phase  I  are  the 
same  industry  partners  who  build  and  deliver  systems  across 
the  federal  government.  Likewise,  many  of  the  PMs  inter¬ 
viewed  had  experience  working  in  other  commodity  domains 
outside  of  space,  representing  civil  agency  and/or  DoD  envi¬ 
ronments.  Therefore,  the  JSCC  study  results  arguably  apply 
to  any  acquisition  domain. 


So  what  does  this  mean  in  terms  of  the  cost  of  EVMS?  Since 
contractors  could  only  provide  level  of  cost  impact  instead  of 
specific  dollar  values  in  the  JSCC  study,  it  is  difficult  to  give 
an  exact  answer.  However,  if  CLT  is  considered  the  "Gold 
Standard"  regarding  the  cost  of  EVMS,  the  JSCC  study  is 
fairly  definitive  as  to  why  the  cost  of  EVMS  on  cost  type 
contracts  should  be  significantly  less  than  the  0.9  percent 
identified  in  1994. 

First,  during  the  1990s,  CLT  based  its  results  on  a  DoD-man- 
dated  C/SCS,  while  JSCC  assumes  that  industry  owns  EVMS 
and  considers  it  to  be  a  best  practice  on  all  types  of  efforts  (not 
just  government  cost-type  contracts).  Second,  CLT  incorpo¬ 
rated  all  aspects  of  the  costs  of  establishing,  maintaining  and 
using  C/SCSC,  while  the  JSCC  study  was  established  to  iden¬ 
tify  the  "delta"  cost  impact  for  EVMS  on  government  versus 
other  contract  efforts  (assuming  the  contractor  already  has 
an  EVMS  management  system  in  place).  This  means  that  the 
cost  impacts  identified  by  the  JSCC  represent  only  a  portion  of 
those  identified  by  CLT.  While  the  JSCC  recognizes  that  there 
is  an  expense  associated  with  designing  and  implementing  an 
EVMS  management  system,  it  is  considered  a  one-time  non¬ 
recurring  expense  that  should  not  be  a  liability  to  the  govern¬ 
ment  since  a  company  should  have  some  type  of  management 
control  system  in  place  to  operate.  In  his  2010  publication, 
Earned  Value  Management:  A  Global  and  Cross  Industry  Perspec¬ 
tive  on  Current  EVM  Practice ,  Dr.  Lingguang  Song  stated  that 
69  percent  of  his  420  studied  groups  voluntarily  used  EVMS. 
This  leads  us  to  think  that  a  growing  A&D  company  will  imple¬ 
ment  EVMS  not  only  to  support  future  government  work,  but 
because  it  is  the  most  prudent  thing  to  do  for  a  self-organizing, 
competitive  and  profit-driven  enterprise. 

The  bottom  line  is  that  there  is  no  smoking  gun  to  show  that 
removing  EVMS  requirements  from  a  government  cost  type 
contract  will  result  in  substantial  cost  savings.  While  the  JSCC 
study  does  show  a  few  higher  EVMS-related  cost  impacts  for  a 
handful  of  programs,  it  does  not  identify  any  systemic  High  or 
Medium  Cost  impacts  that  affect  a  majority  of  the  programs 
that  participated  in  the  survey.  In  almost  every  case  where 
higher  cost  impacts  do  exist,  they  are  typically  driven  by  spe¬ 
cific  contract  requirements. 

When  there  is  suspicion  that  high  EVMS  implementation  costs 
exist  on  a  program,  several  questions  should  be  asked  before 
drawing  any  conclusions: 

Does  the  contractor  use  EVMS  to  manage?  If  so,  is  the  con¬ 
tractor  using  its  management  system  to  support  commercial, 
fixed-price,  or  internal  efforts?  Why  will  it  be  more  expensive 
on  a  cost  type  contract?  Ask  for  specific  details.  Identify  the 
key  cost  drivers  and  obtain  a  basis  of  estimate. 

If  surveillance  is  identified  as  a  key  cost  impact,  what  is 
the  driver?  Before  discussing  the  cost  of  surveillance,  what 
is  a  reasonable  and  appropriate  level  of  corporate  investment 
versus  direct  program  cost  in  the  maintenance  of  EVMS?  Do 
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industry  and  the  government  have  the  same  expectations 
for  EVMS  and  consistent  interpretation  of  the  guidelines? 
Periodic  independent  surveillance  with  timely  resolution  of 
issues  is  part  of  implementing  a  reliable  and  healthy  man¬ 
agement  system. 

If  the  IBR  is  identified  as  a  contributing  cost  impact,  what  is 
the  driver?  How  many  people  will  be  involved  to  support  the 
review?  How  will  their  time  be  spent  differently  from  normal 
day-to-day  program  management  execution?  Developing  a 
program  baseline  is  critical  regardless  of  whether  or  not  an 
IBR  is  scheduled. 

If  there  is  a  request  to  eliminate  the  EVMS  requirement  or 
some  aspect  of  EVM  that  is  already  on  contract,  what  credit 
will  be  given  back  to  the  government?  If  a  specific  require¬ 
ment,  management  activity  or  report  is  eliminated,  will  there 
be  an  actual  reduction  of  cost  and  personnel  on  the  program? 
Are  those  individuals  identifiable  by  name?  If  something  is 
removed  from  a  contract,  the  government  should  expect  to 
pay  less. 


Have  both  government  and  industry  PMs  reviewed  ways 
to  decrease  the  cost  impact  and/or  increase  the  value  of 
EVMS?  Is  the  size  and  number  of  CAs  optimized  for  risk  and 
span  of  control?  Are  PMs  aware  of  additional  costs  created  by 
unique  reporting  requirements?  Have  PMs  read  and  reviewed 
the  results  and  recommendations  of  the  JSCC  study  (available 
at  www.acq.osd.mil/evm/resources/lnitiatives.shtml)?  This 
study  provides  practical  recommendations  and  stakeholder 
actions  to  reduce  costs  and  improve  value,  and  may  be  helpful 
in  identifying  additional  EVMS  efficiencies. 

Admittedly,  using  earned  value  to  manage  a  program  is  not  as 
glamorous  as  flying  jets  or  working  launch  operations.  How¬ 
ever,  the  JSCC  Better  EVMS  Implementation  Study  offers  ob¬ 
jective  evidence  that  when  EVMS  is  properly  maintained  and 
the  data  is  optimally  used,  EVMS  provides  a  high-value  and 
low-cost  management  practice  that  (1)  supports  the  delivery 
of  valuable  systems  to  the  warfighter  and  (2)  helps  protect 
the  American  taxpayer  from  wasteful  spending. 


The  authors  can  be  contacted  at  bembersi@nro.mil;  eknox@tecolote.com; 
jones_michelle@bah.com;  and  jeffrey.s.traczyk@saic.com. 
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A  Practical  PM  Guide  to  Requests 
for  Equitable  Adjustment 


Lt  Col  Scott  K/empner,  USAF 


y  first  experience 
with  a  request  for 
equitable  adjust¬ 
ment  (REA)  was 
brief  and  decisive. 
The  0-6  program  director  didn't  liter¬ 
ally  drop  it  in  the  trash  bin,  but  he  clearly 
wanted  to.  His  message  to  the  develop¬ 
ment  contractor  was  to  not  expect  any 
action  by  the  government,  despite  the 
contractor  fastidiously  mentioning  it 
month  after  month  on  a  chart  listing 
unresolved  contracts  business.  The 
REA  resulted  from  a  technical  dis¬ 
agreement  between  the  contractor 
and  the  government  regarding  how 


Klempner  is  an  Air  Force  Fellow  at  the  Washington  Institute  for 
Near  East  Policy  in  Washington ,  DC.  His  experience  as  an  acguisi- 
tion  program  manager  includes  four  tours  at  the  National  Recon¬ 
naissance  Office ,  the  Air  Force  Space  and  Missile  Systems  Center ; 
and  the  Air  Force  Rapid  Capabilities  Office ,  where  he  developed 
numerous  space  and  airborne  systems ,  payloads ,  advanced  intel¬ 
ligence ,  surveillance  and  reconnaissance  capabilities.  He  also  has 
operational  experience  as  a  Minuteman  III  ICBM  combat  crew 
commander ;  evaluator  and  flight  commander.  He  holds  Level  III 
Defense  Acguisition  certifications  in  program  management  and 
systems  engineering. 
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much  in-scope  testing  was  required  to  properly  resolve  a 
spacecraft  test  fault. 

According  to  Federal  Acquisition  Regulation  subpart  43.2,  a 
contractor  requests  equitable  adjustment— essentially  a  type 
of  proposal— in  response  to  a  unilateral  contract  change  order, 
but  other  unplanned  changes  to  contract  terms,  such  as  a  late 
delivery  of  government  furnished  property  (GFP)  or  disputes 
over  scope,  can  lead  the  contractor  to  send  an  unexpected 
REA.  In  the  daily  life  of  a  program  office,  REAs  are  rare  be¬ 
cause  planned  contract  changes  are  accompanied  by  requests 
for  proposal.  Likewise,  when  the  contractor  and  government 
agree  about  an  unplanned  change,  the  program  manager  (PM) 
would  treat  the  REA  similarly  to  any  other  proposal.  However, 
when  the  REA  results  from  disagreement  on  contract  terms, 
delay  of  work,  or  scope  (either  in  type  or  magnitude),  the  work¬ 
ing  relationship  may  become  tense  if  it  isn't  tense  already.  Both 
the  government  and  contractors  must  weigh  issues  of  fairness 


The  actions  a  contracting  officer  takes  to  respond  to  an  REA 
are  clearly  outlined  by  the  FAR  and  Defense  Federal  Acqui¬ 
sition  Regulation  Supplement,  but  nothing  similar  exists  for 
technical  evaluators.  The  standard  process  for  evaluating  rea¬ 
sonableness  of  proposed  costs  is  meaningless  if  there  is  no 
way  to  analyze  whether  claimed  impacts  were  in  scope  in  the 
first  place.  When  the  first  REA  arrived  from  my  open  architec¬ 
ture  integrator  for  "low-quality  GFP/'  we  looked  for  standard 
guidance  on  how  to  handle  REAs.  Finding  none,  our  team  de¬ 
veloped  a  methodology  to  determine  whether  REA  claims  had 
merit.  Taking  the  contractor's  claim  seriously  and  conducting  a 
dispassionate  analysis  keeps  the  interactions  professional  and 
de-escalates  emotions.  Defining  an  objective  process  upfront 
increases  acceptance  of  the  result  and  perhaps  more  impor¬ 
tant,  it  shows  that  the  government  is  exercising  due  diligence. 

The  process  we  developed  includes  a  flow  chart  (Figure  1) 
and  a  six-step  evaluation  methodology.  It  is  intended  for  PMs 


The  willingness  of  the  associate  contractors  to 
work  with  each  other  would  quickly  degrade  if 


they  distrusted  the  government  to  enforce  the 
assumptions  and  terms  of  each  contract. 


and  duty  to  stakeholders  when  deciding  how  to  proceed.  Deci¬ 
sion  making  may  become  emotionally  charged,  to  the  detri¬ 
ment  of  the  relationship  and  program  progress. 


and  action  officers  conducting  a  technical  evaluation  of  merits 
and  quanta  of  the  claims  and  complements  the  contracting 
officer's  evaluation. 


In  the  situation  described  above,  the  REA  was  a  small  blip 
that  did  not  threaten  the  program's  overall  success— we  had 
an  enormous  cost-plus  satellite  contract  and  recognized  the 
need  for  all  parties  to  work  together  to  get  the  spacecraft  to  the 
launch  pad.  The  issue  slowly  died  and  eventually  went  away. 
In  that  instance,  it  wasn't  a  bad  strategy  for  the  government, 
but  it  was  not  the  ideal  learning  experience  for  a  young  field 
grade  officer  on  how  to  deal  with  the  situation  in  the  future. 

Years  later  I  joined  an  Acquisition  Category  (ACAT)  I  equiva¬ 
lent,  open-architecture  development  program  using  multiple 
fixed-price  contracts  with  interdependent  (but  competing) 
developers.  Team  members  knew  going  in  that  we  had  the 
perfect  environment  for  spawning  REAs.  Not  only  does  the 
government  have  a  duty  to  respond  to  contractor  requests  for 
adjustment,  but  unlike  my  previous  experience  on  the  satellite 
development  program,  here  even  a  modest  REA  had  the  po¬ 
tential  to  derail  the  program.  The  willingness  of  the  associate 
contractors  to  work  with  each  other  would  quickly  degrade  if 
they  distrusted  the  government  to  enforce  the  assumptions 
and  terms  of  each  contract. 


The  six  steps  in  the  REA  evaluation  process  are: 

Stepl:  Establishment  of  facts.  List  all  of  the  claims  made  by 
the  contractor  and  sort  them  into  facts  the  government  agrees 
with  upfront  and  those  which  require  further  substantiation. 
Statements  about  which  the  government  has  no  direct  knowl¬ 
edge  or  a  conflicting  opinion  should  not  be  agreed  to  upfront. 
Usually  the  chain  of  events  can  be  agreed  upon  by  all  parties, 
but  a  claim  that  GFP  was  inadequate  (for  example)  will  re¬ 
quire  supporting  evidence.  It's  the  contractor's  responsibility 
to  provide  such  evidence. 

This  step  forces  the  government  to  articulate  and  understand 
what  exactly  the  contractor  thinks  happened,  what  it  wants, 
and  on  what  grounds.  It  establishes  the  major  issues  of  the 
REA.  It  defines  the  points  the  government  must  address  in  the 
analysis  and  for  which  the  contractor  must  provide  support. 

Step  2:  Examination  of  scope.  The  contract  statement  of  work 
(SOW)  may  or  may  not  be  very  specific.  However,  in  a  scope 
dispute,  all  relevant  paragraphs  must  be  brought  forward  and 
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Figure  1.  REA  Evaluation  Process 
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considered  against  the  claims.  It's  helpful  to  quote  all  relevant 
SOW  language  and  contractual  clauses  directly  in  the  writeup 
to  facilitate  the  work  of  other  reviewers. 

This  is  where  program  management  needs  to  confront  the 
truth  of  how  a  contractor  could  have  ended  up  performing 
out-of-scope  work.  Going  for  a  quick  and  easy  kill  on  scope 
by  broad-brushing  the  topic  will  not  satisfy  anybody,  and  it 
probably  won't  stand  up  to  legal  scrutiny,  if  it  comes  to  that. 

The  evaluator  should  use  the  relevant  contractual  language 
to  conclude  whether  the  work  was  out  of  scope.  If  all  par¬ 
ties  agree  on  this  point,  say  so.  If  not,  the  reviewer  needs  to 
present  a  more  detailed  argument  as  to  why  the  work  was  in 
scope  or  not. 

Sometimes  comparison  with  the  text  isn't  enough.  The  qual¬ 
ity  or  condition  of  GFP  may  not  be  explicitly  defined  in  the 
contract,  but  it's  not  an  excuse  to  stick  the  contractor  with 
the  added  cost  of  dealing  with  unreasonably  low  quality  GFP. 
Contextual  factors  such  as  proposal  assumptions,  reasonable 
person  tests  and  possible  interpretations  should  be  discussed. 

Step  3:  Review  contractual  direction.  A  contractor  cannot 
self-generate  out-of-scope  work.  After  the  contracting  officer 
gives  authority  to  proceed,  there  is  a  presumption  that  all  tasks 
started  are  in  scope.  It  is  critical  to  examine  all  relevant  formal 


and  informal  communi¬ 
cation  between  parties. 
For  the  benefit  of  review¬ 
ers,  list  communications 
such  as  letters  and  emails 
and  summarize  what  was 
said.  Conclude  whether 
the  contractor  requested 
direction  and  if  direction 
was  provided  by  the  con¬ 
tracting  officer. 

Step  4:  Substantiate 
all  claims.  If  it  is  the 

program's  first  REA, 
the  contractor  may  not 
recognize  the  need  to 
provide  any  evidence 
in  support  of  the  REA's 
claims.  My  contractors 
built  REAs  just  like  any 
other  proposal:  They  pre¬ 
dominantly  were  written 
by  the  business  team  fo¬ 
cusing  on  cost  data  and 
pricing  labor  hours,  so 
the  impact  basis  of  esti¬ 
mate  was  well  supported. 
Justifying  the  claim  was 
given  cursory  treatment 
by  contracting  staff,  if  not  ignored  completely.  Resist  the 
temptation  to  handle  this  in  negotiations— making  the  con¬ 
tractor  write  down  its  justification  will  force  it  to  think  matters 
through. 

By  this  point,  looking  at  scope  and  contractual  direction  should 
give  the  action  officer  an  idea  about  where  the  evaluation  will 
end  up,  but  it  is  still  necessary  to  analyze  any  evidence  pro¬ 
vided  by  the  contractor.  Analyze  the  logic  and  applicability  of 
arguments  and  contract  interpretations.  If  the  REA  justification 
is  weak  or  nonexistent,  be  clear  in  the  writeup  about  what  is 
missing. 

Step  5:  Minimization.  The  contractor  has  a  duty  to  minimize 
out-of-scope  work  and  perform  in-scope  work  first.  When  op¬ 
erating  where  REAs  are  being  generated,  the  government  PM 
needs  to  embrace  this  principle— it  provides  the  only  down¬ 
ward  cost  pressure  for  an  REA.  Contracting  normally  relies  on 
competition  or  negotiation  backed  up  by  engineering  expertise 
to  secure  fair  prices  for  the  government,  but  REAs  have  no 
such  protection.  If  the  contractor  allows  out-of-scope,  unnego¬ 
tiated  work  to  occur  in  place  of  negotiated  work  defined  by  the 
SOW  with  the  expectation  that  it  can  be  reimbursed  through 
an  REA,  the  government  has  lost  control  of  the  program. 

Another  consequence  of  minimization  is  that  negotiating  an 
REA  is  not  simply  a  matter  of  negotiating  actuals.  For  example, 
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if  the  contractor  decided  to  perform  the  work  with  Level  5 
engineers  but  could  have  used  Level  3s,  the  government  is 
fully  justified  in  taking  exception.  If  this  were  in-scope  work, 
cost  would  have  been  controlled  first  by  negotiation  and  then 
by  cost  incentives.  With  an  REA,  the  minimization  principle  is 
the  primary  lever. 

From  a  practical  standpoint,  this  step  is  an  extension  of  the 
previous  one.  However,  there  is  value  in  keeping  this  step 
separate  so  that  a  reviewer  can  easily  see  which  costs  were 
substantiated  in  Step  4  and  what  incremental  adjustments 
were  made  in  Step  5. 

Step  6:  Reciprocal  consideration.  If  the  government  has  any 
reciprocal  or  offsetting  equitable  adjustments  against  the  con¬ 
tractor,  this  is  where  positive  and  negative  dollar  amounts  cancel 
each  other  out  to  produce  a  lower  or  zero  net  payment.  Theo¬ 
retically,  the  government  could  press  that  other  claim  against 
the  contractor  separately  and  receive  funding  back  (similar  to  a 
descope  proposal),  but  this  is  so  rare  I  have  never  seen  it  prove 
worth  the  effort.  Despite  this,  the  government  should  never  give 
up  leverage  on  contractor  performance— it  is  still  valuable. 

Where  the  contractor  refuses  to  drop  the  REA,  a  trade  gives 
the  contractor  PM  something  to  sell  to  his  or  her  corporate 
management.  The  trade  doesn't  need  to  be  dollar  exact— 
the  flexibility  afforded  by  negotiations  could  allow  the  con¬ 
tractor  PM  to  make  the  trade  fit  even  when  the  amount 
supported  by  government  analysis  is  lower  than  the  con¬ 
tractor's  original  request. 

Gray  Areas 

The  REAs  my  team  dealt  with  generally  fell  into  two  catego¬ 
ries— some  sort  of  problem  with  GFP  or  proposal  assump¬ 
tions  being  violated.  We  spent  many  long  evenings  weighing 
various  factors  to  determine  how  much  liability  fell  in  the 
government's  corner. 

In  one  case,  our  contractor  started  in-scope  work  and  con¬ 
tinued  working  even  past  the  point  where  the  contractor 
considered  it  to  be  out  of  scope.  The  contractor  received  a 
buggy  GFP  software  delivery  for  integration  into  the  weapon 
system,  but  the  software  code  required  extensive  trouble¬ 
shooting,  repeated  attempts  at  integration,  and  integration 
of  multiple  drops  once  the  software  was  fixed.  Although  the 
work  was  in  scope,  they  made  a  good  point  that  they  didn't 
sign  up  for  unlimited  integration  costs  in  their  fixed-price 
proposal.  Nobody  knew  what  constituted  a  reasonable  upper 
limit,  but  we  all  theoretically  agreed  one  existed.  In  this  case, 
the  auto-generation  principle  decided  the  way  forward:  As 
soon  as  the  contractor  thought  work  was  out  of  scope,  it 
should  have  stopped  and  requested  direction  before  pro¬ 
ceeding.  Finishing  work,  later  deciding  it's  out  of  scope,  and 
submitting  a  REA  is  irresponsible. 

In  another  instance,  low-quality  GFP  also  caused  the  contrac¬ 
tor  to  work  less  efficiently  than  it  had  bid.  We  all  agreed  it 


would  have  been  impractical  to  request  direction.  The  contrac¬ 
tor  had  a  fairly  strong  case  when  this  happened,  except  that 
the  SOW,  not  proposed  price,  determines  the  limits  of  scope. 
To  allow  otherwise  is  to  reward  the  contractor  for  low-balling 
the  bid.  This  is  especially  true  if  the  bid  was  competitive  (it 
was)  and  the  GFP  condition  is  not  documented  in  the  contract 
(it  was  not).  At  the  end  of  the  day,  the  government  met  the 
letter  of  the  contract.  The  argument  was  bolstered  with  an 
"experienced  contractor"  standard— an  experienced  bidder 
should  always  expect  some  level  of  integration  difficulty. 

In  a  final  case,  a  subsystem  provider  underbid  the  amount  of 
integration  support  (software  bug-fixes)  required  for  the  qual¬ 
ity  and  maturity  of  their  offering.  The  contractor  planned  to  do 
this  work  during  system  integration  but  did  not  win  the  inte¬ 
grator  contract,  putting  all  parties  in  an  awkward  position.  In 
pushing  the  contractor  to  comply  with  the  SOW  and  continue 
bringing  the  subsystem  up  to  specification,  we  discovered  the 
practical  limits  of  fixed  price  contracting.  The  contractor  sent 
an  REA  claiming  the  extraordinarily  high  amount  of  support 
required  exceeded  its  interpretation  of  the  SOW.  This  REA 
did  derail  the  program,  and  we  were  at  the  point  of  decid¬ 
ing  between  litigation  and  finishing  the  weapon  system.  The 
government  sustained  the  request  and  finished  the  system. 

The  Big  Picture 

Although  supporting  an  REA  is  disadvantageous  to  the  govern¬ 
ment,  the  objective  of  this  process  is  not  to  summarily  crush 
all  REAs.  It  was  designed  to  produce  a  transparent  position 
all  parties  can  understand.  Sometimes  even  airtight  logic  isn't 
enough  to  satisfy  the  contractor.  They  are  accountable  to  cor¬ 
porate  management,  financial,  and  shareholder  concerns  and 
may  not  be  free  to  simply  drop  an  REA  if  the  corporation  sees 
a  reasonable  chance  of  success.  Although  the  REA  disposition 
is  unilateral,  the  contractor  can  always  initiate  a  legal  claim.  A 
thorough  and  well-reasoned  government  analysis  decreases 
the  likelihood  and  success  of  litigation. 

When  it  comes  to  building  a  weapon  system,  contractor  and 
government  PMs  are  in  it  together.  The  contractor's  decision 
to  send  an  REA  and  the  government's  disposition  both  take 
place  in  the  context  of  the  larger  relationship.  I  have  seen 
government  PMs  give  away  the  farm  in  the  interest  of  main¬ 
taining  a  good  working  relationship,  and  I  have  seen  working 
relationships  degrade  to  the  point  of  yelling  phone  calls  and 
slow  progress.  It's  important  to  navigate  between  extremes 
with  full  understanding  of  the  short-  and  long-term  costs  of  a 
decision  to  support  or  reject  a  contractor's  request  for  equi¬ 
table  adjustment. 

The  views  expressed  in  this  article  are  those  of  the  author  and  do 
not  necessarily  reflect  the  official  policy  or  position  of  the  Air  Force , 
the  Department  of  Defense  or  the  U.S.  Government. 


The  author  can  be  contacted  at  scott.klempner@us.af.mil  or  at 
sklempner@washingtoninstitute.org. 
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Time  after  time,  we  try  to  develop  what  we  think  is  the  next  evolutionary 
leap  forward  in  systems  and  end  up  with  a  product  that  is  a  rather  slight 
improvement  and  not  the  game  changer  we  expected.  Even  more  alarm¬ 
ing,  we  sometimes  lose  sight  of  the  real  need  in  chasing  the  item  itself.  The 
examples  provided  are  chemical  and  biological  protection  systems,  but  the 
concept  is  applicable  across  defense  acquisition.  Sometimes  we  need  to  step  back 
and  take  a  second  or  third  look  at  program  assumptions  and  figure  out  whether  we 
think  what  we  are  doing  makes  sense  in  a  context  larger  than  the  program  itself. 


Slate  is  a  systems  engineer  and  analyst  with  Camber  Corporation  supporting  the  U.S.  Air  Force  Chemical ,  Biological ,  Radiological ,  and 
Nuclear  Defense  Systems  Branch  at  Aberdeen  Proving  Ground  in  Maryland.  O'Connor  is  a  program  manager  and  cyber  operator  for  the  Air 
Force  Pentagon  office  in  Washington  DC  and  a  former  portfolio  manager  for  the  Air  Force  CBRN  Defense  Protection. 


There  is  no  such  thing  as  warfare  without  casualties.  In  years 
past,  in  many  people's  minds,  the  "at-least  unstated"  rule  in 
chemical,  biological,  radiological  and  nuclear  (CBRN)  protec¬ 
tion  was  that  chemical  casualties  were  not  allowed.  This  meant 
protective  gear  and  processes  were  necessarily  bulky  and  hot. 
We  haven't  stopped  and  asked  ourselves,  "How  many  casual¬ 
ties  have  been  caused  by  the  loss  of  operational  effectiveness 
while  wearing  this  gear?"  CBRN  protection  affects  all  the  Ser¬ 
vices  in  different  ways.  This  article  focuses  specifically  on  the 
Air  Force  and  how  requirements  have  traditionally  been  met. 

The  Problem 

While  both  chemical  and  biological  warfare  go  back  a  long  way, 
modern  CBRN  begins  with  the  chemical  warfare  of  World  War 
I.  With  the  advent  of  cyanide  gas  and  nerve  gases,  the  levels  of 
lethality  increased  greatly.  As  a  result,  levels  of  protection  also 
increased.  But  at  what  cost? 

As  an  enterprise,  bulky  and  thermally  burdensome  protective 
garments  have  become  the  norm,  along  with  protective  masks 
that  greatly  restrict  vision  and  head  mobility.  This  protective 
gear  interferes  with  accomplishment  of  mission  objectives.  The 
impact  is  exacerbated  in  warm  to  hot  weather  in  which  there  is 
a  great  reduction  in  the  work  that  can  be  done  before  thermal 
stress  sets  in. 

It  is  time  to  consider  the  future  chemical  and  biological  agent 
protection  that  our  soldiers,  sailors,  airmen  and  Marines  will 
need  and  how  to  provide  that  protection.  Historically,  there 
has  been  an  incorrect  focus  on  what  constitutes  important 
system  requirements.  We've  been  acting  as  if  chemical  and 
biological  protection  is  the  mission  instead  of  understanding 
that  protection  is  only  a  characteristic  that  allows  our  personnel 
to  accomplish  the  mission  under  certain  specific  conditions. 
Phrased  differently,  we  have  been  emphasizing  the  wrong  sys¬ 
tem  requirements. 

When  the  Joint  Service  Aircrew  Mask  (JSAM)  program 
started  more  than  15  years  ago,  it  was  meant  to  be  a  single 
mask  design  for  all  aircraft.  This  lofty  goal  proved  a  bridge 
too  far,  and  only  the  JSAM-Rotary  Wing  (RW)  mask  will  be  a 
system  similar  to  the  original  vision.  The  JSAM-Strategic  Air¬ 
craft  (SA)  mask  will  adapt  a  ground  crew  mask  for  aircrew  use 
but  will  not  provide  the  ability  to  transition  to  and  from  a  fully 
protected  posture  easily  as  originally  hoped.  Finally,  the  pilots 
of  tactical  aircraft  essentially  will  fly  with  the  same  masks  they 
have  used  since  the  1980s,  with  marginal  increases  in  capabil¬ 
ity  and  possibly  greater  limitations  on  combat  effectiveness 
in  certain  scenarios. 

The  Joint  Service-Lightweight  Integrated  Suit  Technology 
(JSLIST)  program  was  a  milestone  in  its  day;  both  for  how  it 
was  managed  and  for  the  testing  methodology  and  technolo¬ 
gies  derived  from  it.  Despite  the  improvements  in  the  acquisi¬ 
tion  and  testing  of  the  new  JSLIST  system,  the  actual  product 
delivered  to  the  warfighter  brought  only  marginal  physiological 
burden  and  mobility  gains.  It  did  introduce  suits  that  could  be 


laundered,  but  20  years  later  we  are  preparing  to  abandon  that 
idea.  In  the  end,  JSLIST  offered  no  significant  improvement  to 
the  warfighter's  operational  capability. 

The  JSLIST  and  JSAM  taught  us  that  systems  acquisition  is 
severely  limited  by  initial  assumptions.  This  is  compounded 
by  acquisition  professionals  lacking  the  properand  appropriate 
testing  protocols,  modeling  and  analysis  to  achieve  the  war¬ 
fighter's  goals.  Continued  testing  followed  that  looked  for  the 
same  thing  each  time  and  only  yielded  minor  improvements 
in  the  systems  fielded.  In  a  sense,  the  old  adage  applies:  "The 
definition  of  insanity  is  to  do  the  same  thing  over  and  over  and 
expect  different  results."  Perhaps  the  most  damning  assump¬ 
tions  are  that  any  "improvement,"  no  matter  how  small,  is  bet¬ 
ter  than  nothing  and  that  we  need  to  field  something  new. 

Twenty  years  of  effort  on  the  JSAM  program  for  tactical  aircraft 
and  the  discussions  involving  the  Uniform  Integrated  Protec¬ 
tive  Ensemble  II  requirements  (the  proposed  replacement  for 
the  JSLIST)  led  to  the  realization  that  the  CBRN  Acquisition 
Community  had  taken  the  wrong  approach  to  CB  protection. 
For  many  years,  program  management  teams  questioned  the 
necessity  of  specific  system  requirements— such  as  the  ability 
of  aircrew  systems  to  survive  a  600-knot  ejection  while  main¬ 
taining  the  same  chemical  protection  as  a  brand-new  system. 
But  it  goes  much  deeper  and  is  more  fundamental.  Someone 
else  (not  the  authors)  recently  said,  "CBRN  is  not  a  mission;  it 
is  an  environment  in  which  we  need  to  perform  the  mission." 
CBRN  defense  is  not  a  mission,  but  it  should  be  an  enabler.  In¬ 
stead,  today  it  is  one  part  enabler  to  two  parts  stumbling  block. 

Mission  Impact 

If  the  disadvantages  are  parceled  out  evenly— when  both  sides 
in  a  conflict  are  subject  to  the  same  burdens  and  disadvantages 
in  using  these  types  of  weapons— the  limitations  previously 
discussed  would  not  pose  such  a  great  problem.  However,  the 
United  States  and  most,  if  not  all,  of  its  allies  eschew  use  of 
this  type  of  weapon.  As  a  result,  the  disadvantages  mostly  are 
one-sided. 

The  focus  of  CBRN  protective  requirements  should  therefore 
be  on  mission  impact.  For  air  power,  the  most  telling  element  of 
mission  impact  is  the  number  of  combat  air  sorties  generated. 
Reducing  combat  air  sorties  by  just  10  percent  has  a  very  real 
and  tangible  battlefield  impact.  Lowering  combat  effective¬ 
ness  to  the  25  percent  to  50  percent  range  severely  hampers 
a  commander's  ability  to  deliver  airpower  when  and  where  it 
is  needed.  But  it  isn't  only  a  matter  of  the  number  of  sorties.  If 
we  also  decrease  the  level  of  mission  effectiveness  or  a  pilot's 
required  endurance  to  conduct  a  sortie,  we  have  magnified  the 
effects  of  reduced  sorties  generation. 

“Pouring  the  Foundation”  for  Requirements 
Generation  and  Analysis 

So,  what  is  required  for  future  CBRN  acquisitions?  We  first 
need  to  realize  the  extremely  low  likelihood  that  we  can  pre¬ 
vent  all  casualties  from  CB  hazards.  This  means  that  an  honest 
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discussion  is  needed  among  stakeholders  to  determine  an  ac¬ 
ceptable  loss  rate.  Acceptable  loss  is  not  limited  to  deaths  but 
also  loss  of  combat  effectiveness,  although  fatalities  should  be 
considered  as  well. 

How  much  reduction  in  effectiveness  can  be  tolerated?  How 
can  we  accept  that  decrease  of  effectiveness  during  the  combat 
sortie  mission,  the  number  of  sorties  that  can  be  generated,  or  a 
mixture  of  both?  What  will  be  the  mode  of  the  decrement— task 
inefficiency  or  casualties?  In  considering  what  level  of  casual¬ 
ties  might  be  "allowable"  (as  opposed  to  "acceptable"),  casual¬ 
ties  come  with  special  costs— emotional  impacts  on  surviving 
crews,  public  reaction  and  the  cost  of  personnel  and  supplies 
needed  to  care  for  the  casualties. 

The  next  need  is  to  understand  the  existing  protective  ca¬ 
pability  of  the  regular  duty  uniform  gear  and  aircrew  flight 
equipment.  Given  current  operational  constructs  and  hazard 
expectations,  what  types  of  casualties  would  be  experienced 
in  the  absence  of  CBRN-specific  protection  equipment?  What 
kind  of  exposure  could  our  personnel  survive  before  moving 
into  the  zone  of  unacceptable  risk?  Next,  could  we  change  our 
tactics  and  procedures  to  reduce  personnel  risk  and  casualties 
to  a  more  acceptable  level?  What  operational  effectiveness  is 
produced  by  these  changes?  Suppose  only  the  normal  duty 
uniform  is  worn,  augmented  by  a  protective  mask? 

Following  this  discussion,  we  need  to  decide  whether  to  pro¬ 
ceed  with  a  material  solution.  Even  if  it  is  determined  that  a 
material  solution  is  needed,  the  results  of  the  analysis  to  the 
aforementioned  questions  of  operational  effectiveness  must 
be  kept  and  used  as  the  baseline  state  to  compare  with  a  new 
material  solution.  When  the  material  development  process 
is  completed,  the  levels  of  operational  risk  and  effectiveness 
should  be  compared  to  the  "baseline"  results  to  determine  if  the 
devised  solution  was  a  success  and  should  be  fielded. 

Protective  Mask  Considerations 

What  are  the  issues  with  protective  masks— in  particular  for 
aircrew  personnel?  The  key  performance  parameters  (KPPs) 
in  the  most  recent  aircrew  capabilities  document  focused  on 
the  chemical  and  biological  protection  afforded  by  mask  and 
filter.  The  masks  provided  the  desired  protection.  However,  the 
problems  posed  for  the  Air  Force  are  not  matters  of  chemical 
or  biological  protection.  Rather,  the  problems  revolve  around 
performance,  including  the  ability  to  prevent  pressure  spikes 
due  to  rapid  decompression  during  descents  from  higher  al¬ 
titudes,  the  masks'  restrictions  on  head  movement  and  the 
wearer's  ability  to  even  see  cockpit  control  displays  as  well  as 
the  outside  environment  and  the  resultant  effect  on  situational 
awareness.  There  also  are  broader  issues  when  the  bulkiness 
of  the  mask  portions  below  the  neck  interfere  with  finding  and 
safely  operating  emergency  controls. 

To  a  lesser  but  very  palpable  degree  for  pilots,  there  is  the  dis¬ 
comfort  of  continually  wearing  the  breathing  mask  sufficiently 
secured  to  provide  protection  during  an  entire  mission  set  in 


aircraft  operating  at  high  acceleration— rather  than  being  able 
to  loosen  the  mask  during  regular,  noncombat  flight.  In  other 
words,  the  mask's  performance  characteristics  can  affect  "fly- 
ability"  and  flight  safety. 

The  rigidness  and  assumptions  of  the  stated  KPPs  drive  design 
elements  to  meet  protection  requirements  at  the  expense  of 
lesser  key  attributes  (the  ability  to  fly  the  plane).  In  the  case 
of  any  other  piece  of  equipment  on  an  aircraft,  these  lesser 
key  system  attributes  are  KPPs.  So  why  is  that  not  the  case  for 
CBRN  aircrew  masks? 

Protective  Clothing  Considerations 

Protective  clothing  is  simpler  to  understand  than  aircrew  pro¬ 
tective  masks.  Beyond  protection  levels,  there  are  three  some¬ 
what  interrelated  primary  concerns  regarding  protective  cloth¬ 
ing:  bulk,  mobility,  and  thermal  burden.  The  fourth  aspect  is 
cost  (which,  though  unstated,  also  is  a  consideration  regarding 
protective  masks).  And  cost  brings  into  play  a  number  of  other 
technical  aspects  such  as  durability,  service  life  and  shelf  life. 

Aircrew  protective  ensembles  are  a  bit  easier  than  those  for  the 
ground  crew  because  mobility  and  durability  concerns  are  less 
strenuous  in  air-crew  clothing.  In  the  case  of  thermal  burden, 
the  chemical  protection  is  not  the  worst  contributor  of  heat 
stress  compared  with  existing  aircrew  life-support  equipment. 
Pilots  have  equipment  layered  on  top  of  their  ensemble  and  a 
good  portion  of  their  torsos  are  covered  by  the  metal,  plastic 
and  fabric  of  the  cockpit  seats.  In  addition,  the  aircrew's  expo¬ 
sure  to  a  threat  that  could  penetrate  the  skin  would  be  greatly 
reduced  by  the  closed  cockpit. 

The  ground  crew  is  a  different  matter.  Here  balancing  protec¬ 
tion  and  other  aspects  is  trickier.  Ground  crews  are  much  like¬ 
lier  to  be  exposed  to  a  potential  threat  over  a  longer  time.  The 
required  range  of  motion  for  ground  personnel  also  is  much 
greater.  And  for  many  specialties,  the  plain,  normal  and  ev¬ 
eryday  physical  hazards  posed  to  the  suits  are  much  greater. 
For  example,  firefighters  deal  with  high  heat  and  flame,  civil 
engineering  personnel  deal  with  rough  material,  and  aircraft 
maintenance  face  the  tight  quarters  and  snag  hazards  found 
in  relatively  small  maintenance  hatches. 

Conclusion 

This  article  has  touched  on  a  number  of  different  concerns, 
but  an  in-depth  analysis  is  needed  to  deliver  the  capabilities 
required  in  future  operations.  This  article  is  offered  to  foster 
discussion  and  generate  thought.  The  last  25  years  have  seen 
only  relatively  small  incremental  improvements  in  CBRN  equip¬ 
ment.  Protection  factors  have  increased,  but  very  little  has  been 
done  to  improve  our  ability  to  "Fly,  Fight  and  Win"  in  a  CBRN- 
contested  environment.  In  fact,  at  this  juncture,  CBRN  protec¬ 
tive  equipment  is  a  mission  hindrance.  It  is  time  to  change  that, 
to  start  over  from  basic  past  assumptions  and  re-evaluate  the 
idea  of  acceptable  risk.  It  is  necessary  that  we  do  so! 

The  authors  can  be  contacted  at  alexander.r.slate.ctr@mail.mil  and 
charles.v.oconnor.mil@mail.mil. 
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"Extreme  remedies  are  very  appropriate  for  extreme  diseases." 


—Hippocrates 


t's  no  secret  that  Department  of  Defense  (DoD)  acquisition  professionals  work  in 
a  very  challenging,  high-pressure  environment.  The  acquisition  process  involves 
an  integrated  product  team  of  diverse  functional  experts  who  must  employ  criti¬ 
cal  thinking  skills,  collaborative  problem-solving  and  robust  communications  to 
be  effective.  This  dynamic  means  that  the  acquisition  team's  behaviors  often  can 
be  critical  factors  in  a  program's  outcome. 

During  a  defense  acquisition  and  industry  career  spanning  more  than  35  years,  I  have  observed  and  participated 
in  both  high-  and  low-performing  acquisition  teams.  The  poor  performing  teams  consistently  adopted  behaviors 
that  I  believe  contributed  to  their  poor  performance  while  the  high  performers  avoided  such  behaviors. 

The  article  identifies  some  poor  team  behaviors  that  should  be  avoided.  Each  of  the  behaviors  is  identified  as  one 
of  the  seven  lethal  acquisition  diseases.  On  the  contrary,  the  behaviors  of  the  high-performing  teams  are  identified 
as  potential  remedies  for  those  afflictions. 

“Throw-it-Over-the-Fence-itis” 

The  throw-it-over-the-fence  disease  involves  a  lack  of  teamwork  and  collaboration.  Developing  acquisition  alterna¬ 
tives,  plans  and  documents  should  involve  a  collaborative  effort  to  get  the  inputs  from  functional  team  members. 
Asking  for  inputs  on  important  documents  with  no  subsequent  dialogue  is  a  symptom  of  this  disease. 


Schultz  is  a  professor  in  the  Defense  Acquisition  University's  Capital  and  Northeast  Region  at  Fort  Belvoir,  Virginia. 
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I  observed  a  classic  example  with  a  contracting  organization  I 
once  worked  with.  This  contracting  organization  required  a  list 
of  completed  documents  before  any  contracting  actions  were 
taken.  The  well-intentioned  rationale  was  that  the  program 
office  needed  to  define  the  technical  scope,  base  line  require¬ 
ments,  and  have  funding  documents  available  to  ensure  that 
the  contracting  officials  were  not  wasting  their  time  before  a 
procurement  package  was  initiated. 

The  problem  was  in  implementation.  The  contracting  staff 
resisted  upfront  dialogue  and  early  planning  discussions  be¬ 
cause  the  checklist  of  completed  documents  was  unavail¬ 
able.  It  eventually  instilled  an  attitude  of  sending  documents 
to  each  other  without  discussing  key  issues,  many  of  which 
had  contractual  implications.  As  a  result,  communications 
and  collaboration  were  stifled  and  teams  neither  effectively 
planned  nor  developed  an  integrated  strategy.  A  senior  leader 
intervention  was  needed  to  stop  this  behavior  and  change  the 
process  since  the  situation  created  an  atmosphere  of  finger 
pointing  and  mistrust  between  the  teams. 

The  remedy?  Program  managers  (PMs)  should  establish  clear 
expectations  for  collaboration  and  team  coordination,  espe¬ 
cially  for  important  program  artifacts.  Not  only  will  the  product 
quality  improve;  the  quality  and  performance  of  the  team  also 
should  be  enhanced.  As  an  example,  in  our  Defense  Acquisi¬ 
tion  University  (DAU)  Services  Acquisition  Workshops,  we 
ask  that  the  contracting  officer,  PM,  subject-matter  experts 
and  other  key  team  members  participate  in  the  entire  event. 

“Ready-Fire-Aimitis” 

This  disease  involves  a  team  that  rushes  through  tasks  without 
completing  key  parts  of  the  task  or  adequate  planning.  This 
is  a  prevalent  disease  because  we  are  tempted  to  avoid  pos¬ 
sibly  tedious  and  time-consuming  in-depth  planning.  A  good 
example  is  not  fully  addressing  some  of  questions  raised  by 
potential  bidders  about  the  draft  Request  for  Proposal  (RFP). 
I  have  observed  teams  answering  such  questions  with  a  mere 
restatement  of  the  draft  requirement.  This  not  only  ignored 
the  question  but  sent  a  message  that  the  company  questions 
were  unimportant.  This  can  result  in  limited  competition,  a 
single  bid  or  even  a  protest.  None  of  these  outcomes  is  good. 

On  the  industry  side,  I  watched  our  business  unit  become  too 
aggressive  in  chasing  new  business.  We  pursued  numerous 
opportunities,  many  of  which  were  low  probability  wins  since 
we  were  either  late  to  the  game  or  did  not  fully  understand  the 
customer  requirements.  After  some  reflection,  we  changed 
the  model  to  establish  more  focused  and  better-informed 
pursuit  decisions.  As  a  result,  our  win  rate  increased  and  our 
business  unit  performance  exceeded  objectives. 

The  PM  can  remedy  this  disease  by  establishing  a  clear  ex¬ 
pectation  of  robust  planning  and  avoiding  the  rush  mentality. 
Moving  forward  from  one  step  to  the  next  involves  successful 
completion  of  necessary  predecessor  events  and  understand¬ 
ing  task  relationships.  In  the  draft  RFP  example,  the  PM  should 


insist  that  all  questions  are  answered  satisfactorily  before  a 
final  RFP  is  developed  and  released.  Establishing  a  culture  of 
high-quality  tasks  and  documents  is  another  good  remedy. 

“Cut-and-Paste-itis” 

Overuse  of  cutting  and  pasting  is  a  symptom  of  the  actual  af¬ 
fliction— a  lack  of  critical  thinking.  While  reusing  some  content 
can  help  save  time  and  capture  complex  concepts,  this  should 
not  serve  as  a  substitute  for  thinking  through  the  problem  and 
developing  content  that  enables  the  best  solution. 

I  worked  in  a  program  office  that  was  accustomed  to  issuing 
sole-source  contracts  to  the  original  equipment  manufacturer. 
This  sole-source  situation  was  driven  by  a  very  complex  and 
unique  weapons  system  configuration  and  lack  of  data  rights. 
When  it  was  time  to  conduct  an  acquisition  strategy,  the  lan¬ 
guage  to  justify  a  sole-source  approach  simply  was  cut  and 
pasted  from  the  last  contract  action  and  was  rarely  ques¬ 
tioned.  After  discussions  with  another  company,  we  decided 
to  challenge  the  status  quo  and  quickly  learned  that  not  every 
modification  to  the  aircraft  needed  to  be  sole  source.  We  ex¬ 
plored  some  limited  competition  and  began  looking  at  actions 
to  migrate  to  a  more  open  architecture,  enabling  even  more 
industry  participation  and  competition. 

The  remedy  for  this  disease?  Apply  a  disciplined  and  rigorous 
thought  process  when  developing  important  work  products  for 
your  program.  Be  careful  to  avoid  cookbook  solutions  that  may 
not  be  the  best  course  of  action  for  your  situation,  even  if  they 
worked  previously.  Conditions  will  change.  Emphasize  critical 
thinking  skills  to  your  team,  and  lead  by  example. 

Critical  thinking  is  one  of  the  fundamental  (and  more  im¬ 
portant)  skills  that  acquisition  professionals  must  employ  in 
order  to  plan  and  execute  programs.  There  is  no  one-size- 
fits-all  approach  to  analyzing  a  program's  unique  aspects  and 
making  informed  decisions.  As  a  DAU  professor,  I  often  see 
students  and  workshop  teams  struggle  when  asked  to  think 
critically.  This  should  not  be  surprising.  Just  like  other  skills, 
critical  thinking  requires  training  and  practical  experience  in 
order  to  achieve  improvement.  Critical  thinking  tools  should 
be  part  of  the  acquisition  team  problem-solving  rhythm  since 
it  is  too  important  to  be  performed  on  an  ad  hoc  basis  or 
ignored.  Numerous  examples  clearly  show  the  benefits  of 
using  these  techniques. 

“Schedule-Driven-itis” 

This  disease  can  be  highly  contagious  in  acquisition  organiza¬ 
tions  because  important  program  milestones  are  highly  visible 
and  have  broad  implications  for  acquisition  outcomes.  "Time 
is  money"  is  a  common  acquisition  saying  and  means  that  we 
incur  additional  costs  as  schedules  slip  to  the  right.  Allow¬ 
ing  time  constraints  to  drive  unreasonable  schedules  is  a  root 
cause  of  the  disease.  But  be  aware  of  the  symptoms. 

I  was  involved  early  in  my  career  with  an  advanced  voice  and 
data  communications  system  program.  This  major  program 
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was  very  visible  and  it  was  imperative  to  get  the  system 
through  development  and  fielding  to  our  joint  warfighters. 
Unfortunately,  we  kept  missing  major  schedule  milestones  due 
to  developmental  issues  and  redesign  work  to  correct  system 
deficiencies  discovered  during  testing.  The  DoD  PM  readily 
accepted  the  contractor's  get-well  plan  and  revised  schedule, 
even  though  the  contractor's  track  record  was  not  credible.  In 
the  end,  the  entire  team,  both  DoD  and  contractor,  lost  cred¬ 
ibility  and  the  program  suffered. 

The  "remedy”  for  this  disease  is  adoption  of  event-driven 
scheduling,  including  robust  schedule  analysis.  An  event- 
driven  schedule  will  assess  a  reasonable  duration  based  on 


I  had  the  opportunity  to  observe  two  users  (the  United  States 
and  an  allied  partner)  of  basically  the  same  system  diverge  in 
developing  a  major  upgrade  to  that  common  system.  While 
the  reasons  for  the  divergence  included  some  good  reason¬ 
ing,  the  end  result  was  two  divergent  fleet  configurations 
with  significant  challenges  to  interoperability,  supportability 
and  affordability. 

How  do  we  overcome  this  disease?  The  remedy  simply  is  to 
foster  a  mindset  of  exploring  possibilities.  We  earlier  con¬ 
sidered  the  necessity  for  critical  thinking.  An  essential  part 
of  critical  thinking  models  is  exploration  of  alternatives.  PMs 
should  keep  an  open  mind  to  solutions  available  from  outside 


We  partnered  with  that  organization  and 
found  additional  opportunities  to  work  together  in 
other  areas,  building  on  the  initial  effort.  We  also 
learned  from  each  other,  making  the  partnership  a 
long-term  win-win  for  both  organizations. 


task  complexity,  resources,  task  dependencies  and  other  rele¬ 
vant  factors  related  to  a  credible  plan  to  complete  the  required 
work  successfully.  An  event-driven  schedule  mentality  rec¬ 
ognizes  that  programs  may  have  compelling  schedule-driven 
milestones  and  that  not  adhering  to  these  risks  significant 
consequences.  However,  the  PM  and  team  should  assess 
and  understand  the  risks  of  signing  up  to  a  schedule-driven 
milestone.  Adopting  an  event-driven  mentality  will  enable  in¬ 
formed  decision  making. 

“Not-Invented-Here-itis” 

While  the  symptoms  vary,  the  root  cause  of  this  disease  is 
resistance  to  change.  In  practice,  not-invented-here  means 
that  an  organization  will  not  use  the  products,  services  or  pro¬ 
cesses  of  others.  The  organization's  leadership  may  believe  it  is 
successful  and  has  no  need  for  outside  help.  The  organization 
also  may  believe  that  the  incorporation  of  external  products  or 
processes  could  introduce  risks  and  even  threaten  the  unit's 
future  workload  or  survival. 

There  are  many  dire  consequences  associated  with  this  dis¬ 
ease,  but  I  believe  the  primary  negative  is  the  duplication  of 
effort  from  trying  to  reinvent  the  wheel.  Imagine  the  cost  and 
schedule  implications  of  developing  a  new  capability  that  is 
already  available  as  a  production  item.  On  top  of  these  costs, 
the  delay  in  providing  the  capability  to  the  warfighter  can  cre¬ 
ate  an  even  greater  problem. 


the  organization.  I  remember  an  air  traffic  management  sys¬ 
tem  we  were  planning  that  used  a  new  sensor  technology. 
The  only  organization  we  could  find  that  had  expertise  in  this 
area  was  outside  of  the  DoD.  We  partnered  with  that  orga¬ 
nization  and  found  additional  opportunities  to  work  together 
in  other  areas,  building  on  the  initial  effort.  We  also  learned 
from  each  other,  making  the  partnership  a  long-term  win-win 
for  both  organizations. 

“Treat-the-Symptom-itis” 

The  "treat-the-symptom”  disease  indicates  poor  risk  manage¬ 
ment  and  reactive  versus  proactive  management.  Managing 
risk  is  clearly  one  of  the  big  areas  of  emphasis  for  DoD  PMs. 
Part  of  risk  management  is  the  development  of  precise  future 
root  cause  statements.  Similarly,  for  issue  management,  PMs 
need  to  identify  the  root  cause,  which  may  not  be  easily  visible 
without  some  examination,  often  using  relevant  data  to  assist. 
This  prevents  using  scarce  resources  to  manage  symptoms 
that  don't  solve  the  root-cause  problem.  I  have  observed  pro¬ 
grams  with  very  elaborate  risk  management  plans  that  were 
nothing  more  than  shelfware.  When  I  asked  why  the  plans 
were  not  used,  the  answer  was  along  the  lines  of  "we  don't 
need  to  use  it  or  don't  have  the  time  and  resources  for  this 
small  program.” 

As  an  example,  I  exercised  some  oversight  of  a  team  that  had 
a  great  track  record  in  acquiring  and  deploying  production 
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radars  in  diverse  environments  for  foreign  military  sales  cus¬ 
tomers.  Given  their  past  efforts,  why  should  they  waste  time 
worrying  about  risks? 

This  same  team  subsequently  had  a  major  customer  express 
dissatisfaction  with  the  performance  of  their  newly  installed 
radar.  The  surveillance  coverage  was  very  limited  and  did  not 
detect  aircraft  of  interest.  It  did  not  matter  to  the  customer 
that  the  mountainous  terrain  blocked  the  view  of  the  sensor; 
the  program  office  needed  to  fix  the  problem. 

In  hindsight,  robust  risk  management  could  have  helped  the 
team  identify  the  terrain  as  a  limiting  factor  in  surveillance 
coverage.  As  a  mitigation,  the  team  should  have  assessed  the 
siting  options,  using  models  to  better  understand  the  various 
alternatives,  working  with  the  customer  before  installation  to 
plan  for  the  best  option.  This  could  have  helped  avoid  a  costly 
redeployment  of  the  radar  to  a  more  favorable  site. 

“Conspiracy-of-Hope-itis” 

This  disease  is  associated  with  overly  optimistic  planning  as¬ 
sumptions  about  a  team's  ability  to  execute  a  program  within 
cost,  schedule  and  performance  constraints.  Some  common 
symptoms  include  poor  program  start-up  planning,  lack  of 
analytical  rigor  and  heavy  reliance  on  contractor  sales  pitches 
of  program  possibilities. 

Several  motivations  make  this  disease  prevalent  and  hard  to 
prevent.  These  include  the  "can-do”  attitude  of  high-perform¬ 
ing  teams;  a  desire  to  work  on  high-visibility  programs;  the 
wish  to  keep  a  team  or  organization  employed  and  to  obtain 
or  justify  funding,  and  many  others  factors.  PMs  must  be  very 
careful  with  this  disease,  which  can  lead  to  significant  nega¬ 
tive  outcomes. 

An  example  of  a  personal  experience  with  this  disease  involved 
a  significant  development  effort  for  a  command  and  control 
system.  This  occurred  in  the  days  of  acquisition  lightning  bolts 
that  in  part  called  for  smaller  DoD  acquisition  program  of¬ 
fices  with  greater  reliance  on  contractor  expertise  to  manage 
the  cost-performance  aspects  of  a  program  through  its  life 
cycle.  In  this  case,  the  PM  and  the  contractor  were  extremely 
optimistic  that  the  joint  team  could  deliver  on  a  very  complex 
development  with  minimal  help  from  the  DoD. 

It  turned  out  that  the  DoD  program  office  did  a  poor  job  plan¬ 
ning  for  the  contract's  scope  and  complexity.  Furthermore, 
the  program  office,  with  its  limited  team,  could  not  effectively 
manage  the  cost-performance  trades  and  keep  up  with  design 
review  and  approvals,  and  this  affected  the  contractor's  ability 
to  execute  in  a  tight  schedule.  The  contract  suffered  significant 
cost  growth  and  had  to  be  renegotiated  to  incorporate  a  more 
realistic  program.  Several  years  and  many  millions  of  dollars 
later,  the  program  recovered,  but  the  optimism  lesson  lingers. 

Treating  this  optimism  disease  involves  a  more  complex  di¬ 
agnosis  because  the  root  cause  needs  to  be  diagnosed  be¬ 


fore  there  can  be  any  treatment.  Treatment  can  range  from 
improved  program  start-up  planning  and  cost  estimating  to 
ethics  training. 

Additional  Treatment  Considerations 

Building  an  organizational  environment  of  trust,  empowerment 
and  integrity  helps  lay  a  foundation  to  avoid  these  diseases. 
PMs  and  their  program  office  leadership  team  must  lead  by 
example  in  this  area  and  ensure  that  staff  members  are  em¬ 
powered  to  speak  up  without  fear  of  incurring  retribution  or 
criticism.  This  type  of  environment  is  crucial  to  effective  critical 
thinking;  people  must  be  open  to  sharing  ideas  and  feel  that 
their  input  is  valued. 

Some  of  these  PM  leadership  qualities  can  be  associated 
with  soft  skills  such  as  emotional  intelligence,  critical  think¬ 
ing,  change  leadership,  coaching,  mentoring,  and  managing 
conflict.  Many  training  courses  and  workshops  are  available 
to  develop  these  skills,  including  those  at  DAU.  Most  of  these 
workshops  can  be  highly  tailored  to  address  the  specific  areas 
of  concern  and  offer  a  great  opportunity  for  the  acquisition 
team  to  reflect  on  specific  actions  that  can  improve  perfor¬ 
mance.  PMs  should  consider  these  training  opportunities  and 
also  consider  intact  team  efforts  to  get  the  best  return  on  the 
training  investment. 

There  always  is  a  danger  that  the  seven  lethal  diseases  will 
become  resistant  to  treatment.  It  is  important  to  identify  the 
root  causes  and  ensure  the  affliction  is  treated  promptly  so  it 
does  not  get  worse.  While  treatments  usually  are  effective, 
good  preventive  maintenance  always  is  the  best  approach 
because  it  can  help  avoid  the  diseases  altogether!  & 

The  author  can  be  contacted  at  brian.schultz@dau.mil. 


MDAP/MAIS  Program 
Manager  Changes 


With  the  assistance  of  the  Office  of  the  Secretary  of 
Defense,  Defense  AT&L  magazine  publishes  the  names 
of  incoming  and  outgoing  civilian  and  military  program 
managers  for  major  defense  acquisition  programs 
(MDAPs)  and  major  automated  information  system 
(MAIS)  programs.  One  change  of  leadership  was 
reported  for  September  and  October  2016. 

Navy/Marine  Corps 

CAPT  Laura  Schuessler  relieved  CAPT  Scott  Porter  as 

program  manager  for  the  Integrated  Defensive  Electronic 
Countermeasures  System  (ALQ-214)  Program  (PMA 
272)  on  Oct.  14. 
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Requirements  Management 

The  Need  to  Overhaul  JCIDS 


Thomas  H. 


Acquisition  reform  continues  to  receive 
a  great  deal  of  attention  from  both  the 
Senate  and  House  Armed  Service  Com¬ 
mittees.  Reform  initiatives  to  date  tend 
to  focus  exclusively  on  the  "little  a"  puz¬ 
zle  piece  of  the  defense  acquisition  process— i.e., 
the  Defense  Acquisition  System  (DAS). 

The  other  two  pieces  of  the  "big  A"  process— the  requirements  system  (Joint  Capabili¬ 
ties  Integration  and  Development  System  [JCIDS])  and  the  financial  system  (Program 
Planning  Budgeting  and  Execution  [PPBE]) — to  date  have  had  a  free  pass  from  Congress; 
yet  these  processes  share  a  good  portion  of  the  blame  for  continuing  poor  results  from 
acquisition  programs. 

The  JCIDS  process  in  particular  requires  a  complete  overhaul  because  it  is  too  bureaucratic 
and  cumbersome  to  keep  up  with  the  speed  of  the  current  information  age  technology 
development  cycle.  It  has  been  assessed  by  Bill  Greenwalt  of  the  American  Enterprise 
Institute  as  "one  of  the  few  processes  that  is  even  more  dysfunctional  than  the  acquisition 
process";  and  described  by  other  authors  as  "byzantine"  and  "one  of  the  most  inscrutable 
strands  of  Pentagon  red  tape."  The  Joint  Chiefs  of  Staff  offices  that  run  the  process  are 
overstaffed,  inefficient,  and  require  major  streamlining  to  become  more  responsive  and 
effective.  Michele  Flournoy,  former  Under  Secretary  of  Defense  for  Policy,  said  that,  "The 
Joint  Staff  and  the  Office  of  the  Chairman  have  grown  to  nearly  4,000  people  ...  the  staff 
should  be  smaller  and  more  focused  on  providing  advice  to  the  president,"  noting  that 
bloated  headquarters  staffs  "undermine  both  performance  and  agility." 

JCIDS  was  implemented  in  2003,  at  the  direction  of  then  Secretary  of  Defense  Donald 
Rumsfeld,  with  the  intention  that  it  should  emphasize  joint  requirements  development 
and  in  order  to  establish  an  analytical  process  for  identifying  potential  material  and  non¬ 
material  solutions  for  validated  capability  gaps.  The  Defense  Acquisition  Portal  further 
explains  the  purpose  of  JCIDS:  "The  JCIDS  process  exists  to  support  Joint  Requirements 
Oversight  Council  (JROC)  and  Chairman  of  the  Joint  Chiefs  of  Staff  (CJCS)  responsibilities 
in  identifying,  assessing,  validating,  and  prioritizing  joint  military  capability  requirements. 
JCIDS  provides  a  transparent  process  that  allows  the  JROC  to  balance  joint  equities  and 
make  informed  decisions  on  validation  and  prioritization  of  capability  requirements."  Yet, 
both  the  Congress  and  its  investigatory  arm,  the  Government  Accountability  Office  (GAO), 
have  questioned  whether  the  system  is  effective  in  meeting  joint  force  needs;  and  the  DoD 
acknowledged  to  the  GAO  in  2012  that  JCIDS  has  been  ineffective  in  helping  the  JROC 
carry  out  its  responsibilities.  What  are  the  specific  problems  associated  with  the  JCIDS 
process,  and  how  can  they  be  fixed  to  produce  a  nimble  and  responsive  requirements 
development  system  that  still  supports  the  JROC's  Title  10  responsibilities  to  help  the 
CJCS  (1)  identify,  assess  and  improve  joint  military  requirements;  (2)  establish  and  assign 
priority  levels  for  joint  military  requirements;  (3)  review  the  estimated  levels  of  resources 


Miller  is  on  independent  defense  acquisition  and  contracting  consultant.  He  is  the  former  program  manager 
for  Mine  Resistant  Ambush  Protected  ( MRAP )  vehicles  within  the  U.S.  Marine  Corps'  Program  Executive  Of¬ 
fice  for  Land  Systems  and  is  a  former  assistant  program  executive  officer  and  U.S.  Army  contracting  officer. 
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required  to  fulfill  joint  requirements;  and  (4)  make  trade-offs 
between  cost,  schedule  and  performance  constraints? 

The  following  is  a  short  summary  of  the  problems  that  I  see, 
followed  by  a  very  compressed  overview  of  possible  steps  to 
make  the  JCIDS  process  more  effective. 

Problem  1:  The  JCIDS  process  is  too  slow  and  bureaucratic. 
The  process  is  extremely  complex  and  requires  a  long  series 
of  successive  "heel  to  toe"  steps  intended  to  ensure  that  the 
analytical  and  staffing  elements  of  the  process  are  followed 
and  fully  documented.  "Capability  sponsors"  (generally,  the 
requirements  organizations  within  the  Services,  such  as  the 
Army's  Training  and  Doctrine  Command  [TRADOC]),  are 


requirements  documents.  Other  changes  limited  page  lengths 
to  force  concise  documents;  established  staffing  targets  of  83 
(later  97)  days  for  the  deliberate  process  and  15  to  31  days  for 
the  urgent/emergent  process.  Other  changes  created  a  more 
robust  "tripwire"  process  that  requires  sponsors  to  return  to 
the  JROC  for  Cost,  Schedule  and/or  Performance  slips. 

While  laudable,  it  is  doubtful  that  these  changes  actually  re¬ 
duced  the  estimated  15  to  20  months  required  to  gain  final 
approval  of  a  requirements  document.  The  complex  process 
continues  to  drive  increased,  time-consuming  analysis,  re¬ 
views  and  staffing  within  the  Services  before  they  submit 
the  requirements  documents  into  KM/DS  for  JS  review.  In 
addition,  documents  frequently  are  returned  to  the  Service 
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The  expedited  review  process  for  Joint  Urgent  Operational  Needs  Statement /Joint 


Emergent  Operational  Needs  is  a  good  initiative,  but  very  few  documents  will  go  down 
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these  lanes'.  Better  to  create  a  single  process  that  expedites  review  and  approval  of 
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required  to  create  their  requirements  documents  in  JCIDS- 
standard  format,  gain  endorsement  from  Joint  Staff  (JS)  of¬ 
fices  on  applicable  content  (e.g.,  J4  endorsement  of  the  Energy 
key  performance  parameter  [KPP]),  and  then  submit  the  docu¬ 
ments  to  the  J8  (Force  Structure,  Resources,  and  Assessment 
Directorate)  "Gatekeeper"  through  the  JS  mandatory  require¬ 
ments  document  database,  Knowledge  Management  and  Dis¬ 
tribution  System  (KM/DS).  In  addition,  there  are  several  levels 
of  review,  including  Functional  Capabilities  Board  (FCB)  and 
Joint  Capabilities  Board  (JCB)  reviews,  before  the  document 
even  makes  it  to  the  JROC.  This  is  particularly  true  for  require¬ 
ments  documents  (primarily  those  that  lead  to  Acquisition 
Category  (AC AT)  ID  programs)  that  are  designated  as  "JROC 
Interest"  and  that,  therefore,  require  JROC  approval. 

The  JS/JROC  staffing  and  review  process  is  performed  in  ad¬ 
dition  to  (and  generally  duplicative  of)  the  staffing  and  review 
process  within  the  sponsoring  Service.  In  2010,  Gen.  James 
("Hoss")  Cartwright  initiated  an  end-to-end  review  of  JCIDS 
in  order  to  improve  the  process'  responsiveness  and  decision 
support  to  the  JROC.  This  resulted  in  major  rewrites  of  Chair¬ 
man  Joint  Chiefs  of  Staff  Instruction  (CJCSI)  5123.01— the 
JROC  Charter,  CJCSI  3170.01— JCI DS,  and  the  JCIDS  Manual 
in  Jan  2012.  Changes  to  the  process  included  creating  three 
potential  process  "lanes"  for  different  circumstances— the 
traditional  process  for  deliberate  requirements  documents, 
and  streamlined  review  processes  for  emergent  and  urgent 


sponsor  by  the  J8  Gatekeeper  for  revisions,  often  for  format¬ 
ting;  restarting  the  staffing  clock.  Finally,  the  expedited  review 
process  for  Joint  Urgent  Operational  Needs  Statement /Joint 
Emergent  Operational  Needs  is  a  good  initiative,  but  very  few 
documents  will  go  down  these  lanes.  Better  to  create  a  single 
process  that  expedites  review  and  approval  of  all  requirements 
documents. 

Problem  2:  Hierarchical  review  slowed  by  a  bloated  JS  orga¬ 
nization.  The  JCIDS  process  is  hierarchical,  with  requirements 
and  approvals  flowing  from  the  JROC  back  to  the  Service  spon¬ 
sors.  The  sponsor  organizations  have  very  little  influence  on 
the  process,  other  than  through  the  "Old  Boy  Network"  (i.e., 
some  General  Officers  who  seek  to  influence  their  peers  on 
the  JROC  and/or  JCB).  This  in  itself  defeats  the  purpose  of 
independent  review. 

This  process  has  driven  a  significant  increase  in  the  JS  military 
and  civilian  organization  structure  (the  overall  organic  govern¬ 
ment  personnel  on  the  JS  exceed  4,000,  with  an  unknown— 
but  probably  much  larger— number  of  support  contractors). 

Many  of  these  people— including  the  members  of  the  JCB  and 
JROC— lack  the  technical  expertise  and  experience  needed  to 
fully  understand  the  requirements  in  the  documents  (the  GAO 
noted  that  the  House  Armed  Services  Committee  "received 
testimony  that  the  Joint  Staff  lacked  some  of  the  analytical  ex- 
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pertise  necessary  to  ensure  that  the  JCI DS  process  rigorously 
vets  proposed  requirements  ...  and  we  noted  that  capability 
needs  continued  to  be  proposed  and  defined  by  the  Services 
with  little  involvement  from  the  joint  community")-  And  the 
personnel  generally  are  not  trained  in  the  acquisition  process. 
All  of  these  factors  combine  to  create  an  overly  bureaucratic, 
complex  process  that  involves  excessive  reviews  by  multiple 
layers.  This  ultimately  slows  the  "big  A"  acquisition  process 
with  little  real  return  on  this  investment  in  time. 

Problem  3:  Multiple  analytical  reviews  and  KPPs  that  drive 
cost  and  limit  trade  space.  The  hierarchical  JCI  DS  process- 
driven  by  a  risk-averse  culture's  need  for  top-down  control— 
has  resulted  in  establishment  of  a  significant  number  of  man¬ 
datory  KPPs.  The  "AcqNotes.Com"  website  defines  a  KPP  as 
"key  system  capabilities  that  must  be  met  in  order  for  a  system 
to  meet  its  operational  goals." 

KPPs  are  intended  to  be  kept  to  the  absolute  minimum  nec¬ 
essary  to  ensure  operational  effectiveness,  in  order  to  allow 
maximum  flexibility  so  the  program  manager  can  seek  the 
most  technically  capable,  affordable  material  solution  and 
propose  appropriate  trade-offs  with  requirements  (KPPs 
aren't  tradeable). 

Multiple  mandatory  KPPs  significantly  reduce  that  flexibil¬ 
ity,  increase  cost,  and  limit  access  to  technical  solutions  that 
may  provide  greater  overall  capability.  There  are  currently  six 
JCIDS-directed  mandatory  KPPs:  (1)  Force  Protection;  (2)  Sys¬ 
tem  Survivability;  (3)  Sustainment;  (4)  Net-Ready;  (5)  Energy; 
and  (6)  Training.  All  of  these  KPPs  cause  the  Service  sponsors 
to  spend  significant  time  and  resources  on  analytical  efforts 
for  each  KPP.  Moreover,  review  and  endorsement  of  each  KPP 
must  be  coordinated  with  the  applicable  JS  office. 

In  addition,  JCI  DS  mandates  a  torturous  capability  gap  analy¬ 
sis  process  prior  to  creation  of  the  first  requirements  docu¬ 
ment— the  Initial  Capability  Document  (ICD) — culminating  in  a 
Capabilities  Based  Assessment  (CBA).  This  analytical  process 
extends  document  processing  time  and  consumes  significant 
resources,  often  into  the  millions  of  dollars  (many  studies  are 
contracted  out,  of  course).  But,  again,  this  adds  little  value 
to  defining  the  requirements  for  the  ultimate  material  solu¬ 
tion.  As  Dr.  Michael  Cochrane  has  written:  "The  problem  is 
that  the  capabilities-based  reality  has  never  quite  lived  up  to 
the  capabilities-based  theory  ...  so-called  'gap  analyses'  are 
nothing  more  than  highly  subjective,  qualitative  statements ... 
there  is  nothing  rigorous  or  analytical  about  this,  so  why  beat 
around  the  bush?  If  the  joint  force  commander  wants  more  Y, 
just  ask  for  more  Y!" 

Problem  4:  Too  many  requirements  documents.  Preparing 
the  three  requirements  documents— the  ICD,  the  Capability 
Development  Document  (CDD),  and  the  Capability  Production 
Document  (CPD)— requires  extensive  analysis  and  resources. 
All  three  require  separate,  lengthy  staffing/review  cycles— 
again  slowing  the  overall  "Big  A"  acquisition  process.  The  idea 


behind  having  three  separate  documents  is  that  the  required 
capabilities  are  defined  and  documented  in  increasing  detail  as 
knowledge  is  gained  while  the  acquisition  program  progresses 
through  its  life  cycle.  However,  little  additional  knowledge  is 
gained  from  developing  and  staffing  the  ICD  and  CDD— at 
least  not  enough  to  justify  the  resources  and  time  required  to 
staff  and  prepare  the  documents.  Regardless  of  the  theory, 
the  Service  sponsors  generally  know  what  material  solution 
they  want  at  time  of  ICD  (and  earlier).  Working  with  their  ac¬ 
quisition  counterparts,  the  sponsors  conduct  enough  market 
research  to  know  the  capability  of  the  potential  systems  or 
technologies  available  in  the  marketplace. 

Problem  5:JS/JROC  review  adds  little  value.  The  vast  major¬ 
ity  of  the  documents  reviewed  by  the  JROC  and  subordinate 
boards  are  approved  without  comment.  This  is  due  to  several 
of  the  reasons  cited  above,  including  lack  of  subject-matter 
expertise  on  the  JS  and  a  collegial  culture  that  discourages 
JCB/JROC  general  officers  from  disapproving  or  changing  re¬ 
quirements  put  forward  by  their  peers.  Very  few  of  the  docu¬ 
ments  result  in  JROC  memorandums  that  direct  changes  in 
requirements  based  on  trade-offs  among  cost,  schedule  and 
performance  constraints,  or  that  direct  the  Services  to  seek 
joint  material  solutions.  The  JCI  DS  process  therefore  results  in 
spending  vast  resources  and  slowing  the  acquisition  and  field¬ 
ing  of  military  equipment.  But  based  on  the  low  percentage  of 
documents  disapproved  or  changed,  this  work  provides  a  very 
poor  return  on  this  investment  in  funds  and  time. 

In  summary,  the  JCI  DS  process'  multiple  steps  and  multiple  re¬ 
view  layers  slow  requirements  document  validation  to  a  crawl, 
while  the  process  itself— as  well  as  the  large  JS  organization 
that  sustains  it— adds  little  toward  providing  the  warfighters 
with  better  and  faster  technical  capabilities.  The  Service  spon¬ 
sors  spend  months  and  often  millions  of  dollars  complying 
with  the  "byzantine"  JCI  DS  process,  but  at  the  end  the  vast 
majority  of  the  documents  are  "rubberstamped"  in  the  JS/ 
JROC  review. 

When  he  initiated  his  JCIDS  review  initiative,  Cartwright 
said:  "We're  starting  to  rewrite  JCIDS.  It  has  been  gamed 
to  death  and  we're  going  to  throw  it  away.  Unfortunately, 
they  didn't  do  that.  Instead,  they  have  initiated  changes  that 
double  down  on  the  current  model;  primarily  because  the  JS 
JCIDS  team— like  the  proverbial  fox  guarding  the  hen  house- 
implemented  the  changes.  I  believe  that  the  DoD  (and  the 
taxpayer)  can  get  better  value— and  a  much  more  stream¬ 
lined  requirements  review— by  taking  a  few  simple  steps  to 
streamline  the  process: 

■  Reduce  JS/JROC  involvement  to  the  bare  minimum  required 
to  meet  the  intent  of  Title  10  responsibilities  and  delegate 
the  remaining  authority  to  the  Services. 

■  Require  JROC  review  of  requirements  documents  only  for 
Major  Defense  Acquisition  Programs  (MDAPs),  do  away 
with  the  FCB  and  JCB  pre-reviews,  and  delegate  all  other 
reviews/approvals  to  the  Services. 
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■  Conduct  periodic  JS/JROC-led  portfolio  reviews  that  focus 
on  ensuring  that  the  Services  pursue  joint  material  and 
non-material  solutions  to  capability  requirements,  and  that 
overall  requirements  are  aligned  to  strategic,  Combatant 
Commander,  and  budgetary  priorities. 

■  The  Under  Secretary  of  Defense  for  Acquisition,  Technol¬ 
ogy,  and  Logistics  and  the  Under  Secretary  of  Defense 
(Comptroller),  and  JS  Vice  Chairman  co-chair  both  JROC 
and  Acquisition  Milestone  reviews,  ensuring  active  coordi¬ 
nation  and  trade-off  agreements  between  the  three  "circles" 
of  the  "Big  A"  Acquisition  process. 

■  Allow  the  Services  tailoring  of  the  analytical  requirements— 
particularly  CBAs— to  the  program  requirements. 

■  Replace  the  current  three  requirements  documents  with  a 
single  common  document,  called  a  Capability  Requirements 
Document  (CRD),  that  is  approved  once  prior  to  Milestone 
B,  and  only  updated  thereafter  as  required  by  fact-of-life 
changes. 

■  Eliminate  all  mandatory  KPPs,  and  make  the  current  six  (6) 
mandatory  KPPs  optional  for  use  by  the  Service  sponsors 
as  applicable  to  the  specific  acquisition  program. 

■  Significantly  reduce  the  number  of  personnel  in  the  JS  orga¬ 
nization  that  support  the  JCIDS  and  JROC  to  a  core  profes¬ 
sional  staff  that  is  fully  trained  and  certified  in  Requirements 
Management,  Acquisition,  and  Financial  Management. 


The  goal  of  these  changes  is  to  simplify  the  overall  process, 
push  decision  authority  down  to  the  level  appropriate  for  a 
risk-informed  culture,  reduce  overall  cycle  time,  reduce  re¬ 
quirements  development  costs,  and  allow  the  JS/JROC  to 
focus  on  their  core  missions  of  providing  advice  to  the  presi¬ 
dent  and  (at  a  much  higher  level)  prioritizing  joint  military 
capability  requirements. 

The  current  JCI DS  process  is  typical  of  the  top-down  Pentagon 
hierarchical  control  processes  in  force  since  World  War  II.  Un¬ 
fortunately,  the  process  moves  too  slowly  to  keep  up  with  the 
light-speed  pace  of  the  evolving  threats  that  it  is  intended  to 
counter  and  much  too  slow  to  acquire  and  field  the  technolo¬ 
gies  required  to  fill  21st  century  capability  gaps.  (As  noted  by 
former  Air  Force  Vice-Chief  of  Staff  for  Intelligence  Lt.  Gen. 
David  Deptula,  "A I  Qaeda  doesn't  have  a  JCIDS  process ...  we 
need  to  be  able  to  operate  much  quicker  and  inside  our  ad¬ 
versary's  decision  loop.")  In  order  to  make  the  process  more 
effective  in  terms  of  staying  ahead  of  the  threat  curve,  the  DoD 
will  need  to  sacrifice  control  for  speed  by  allowing  the  Services 
more  authority  to  seek  innovative  solutions  and  to  more  rap¬ 
idly  acquire  new,  cutting-edge  technologies  (both  hardware 
and  software)  and  get  them  in  the  hands  of  the  warfighter.  & 

The  author  can  be  contacted  at  millerth3@gmail.com. 
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to  share  BBP  knowledge  and  experience 

DEFARTMENTOP  DEFENSE 

— /fMM — — — - _ _ _  -  w  jkr-  -3 
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Take  a  Deep  Dive  With  DAU 

The  Kind  We  Offer  Is  Safe  and  Sure  to  Keep  You  Dry 


Woody  Spring  ■  Rebecca  Haydu-Jackson 


A  technical  deep  dive  for  a  submarine  may  involve  carefully  character¬ 
izing  submersion  depths  that  approach  the  performance  limits  of 
hull  integrity  or  other  system  limitations.  For  a  scuba  diver,  a  Deep 
Dive  could  be  better  understanding  the  capabilities  and  limitations 
of  human  performance  or  the  depth  at  which  nitrogen  narcosis 
begins  to  set  in. 

Spring,  a  graduate  of  West  Point  and  former  NASA  astronaut currently  works  at  the  Defense  Acquisition  University  West  Region  in  San 
Diego ,  California,  as  a  professor  of  Engineering,  Test  and  Evaluation,  as  well  as  chief  learning  officer  and  an  executive  coach.  Haydu-Jackson 
is  a  former  contractor  supporting  DAU  West  Region.  She  completed  her  coursework  toward  a  master's  degree  in  business  administration 
and  has  since  left  DAU  for  other  employment. 
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Figure  1.  The  DAU  Acquisition  Learning  Model 


Foundational  Learning 

Deliver  high-quality,  cur¬ 
rent,  easily-accessible  ac¬ 
quisition  training,  providing 
long-term  knowledge  and 
critical  thinking  skills. 


Workflow  Learning 

Provide  customer-focused 
learning  assets  and  job 
support  tools,  at  the 
point  of  need,  to  help  the 
Defense  Acquisition  Work¬ 
force  succeed. 


Performance  Learning 

Conduct  wide  range 
of  workplace  support 
including  high-impact 
customer  engagement  to 
help  improve  acquisition 
outcomes. 


DAU  professor  who  describes  the  ALM  either 
in  class  or  in  a  conversation  afterward. 

Can  DAU  Give  Us  What  We  Need? 

DAU  usually  conducts  a  series  of  initial  in¬ 
terviews  with  leadership  at  multiple  levels 
to  help  determine  the  most  suitable  type  of 
MA.  While  a  crucial  component  for  many  of 
the  MA  solutions  at  our  customers'  disposal, 
interviews  alone  are  generally  not  sufficient 
to  determine  if  a  deep  dive  is  well  suited. 
After  a  series  of  more  intensive  discussions 
with  DAU,  it  might  prove  to  be  a  good  fit.  We 
have  learned  that,  if  we  start  the  MA  only 
after  interviews,  we  are  often  addressing  or 
fixing  symptoms  and  not  the  root  cause.  If  we 
do  not  address  the  root  cause,  the  symptom 
usually  will  reappear  in  the  same  or  mutated 
condition  and  still  require  MA,  or,  later,  inter¬ 
vention  to  correct. 


Outside  the  aquatic  world,  the  term  "deep  dive"  is  more  of 
a  metaphor;  it  is  used  in  many  disciplines  and  has  various 
meanings  depending  on  the  domain  using  the  term.  For  ex¬ 
ample,  a  technical  deep  dive  may  involve  an  evaluation  of 
the  underlying  scientific  basis  of  a  technical  pursuit,  or  an 
in-depth  analysis  of  a  program's  technical  maturity,  or  a  rigor¬ 
ous  engineering  review  of  the  current  application  of  certain 
technology  vital  to  a  program's  success.  A  business  office 
might  even  use  a  deep  dive  technique  for  the  divergent  part 
of  a  brainstorming  session. 

As  part  of  its  Mission  Assistance  (MA)  portfolio  of  products 
and  services  available  to  the  Defense  Acquisition  Workforce, 
the  Defense  Acquisition  University  (DAU)  has  been  conduct¬ 
ing  deep  dives  for  more  than  a  decade.  Our  customers  tell 
us  that  this  in-depth  analysis  of  their  organization  has  helped 
them  better  understand  inherent  challenges  and  exposed 
some  of  the  associated  root  causes  inhibiting  the  achievement 
of  more  successful  acquisition  outcomes. 

Where  Deep  Dives  Fit  Into  DAU’s  Overall 
Learning  Architecture 

Figure  1  presents  DAU's  Acquisition  Learning  Model  (ALM). 
The  right-side  diamond  shows  Performance  Learning,  or  the 
learning  done  in  the  workplace,  and  three  of  the  competencies 
that  support  Performance  Learning  written  on  the  diamond 
perimeter.  The  supporting  competencies  are  Demonstrated 
Experience,  Workshops,  and  Mission  Assistance.  Research 
has  shown  that  almost  80  percent  of  all  learning  takes  place  in 
the  workplace  and  that's  where  MA  comes  in— the  workplace 
and  not  necessarily  the  classroom  or  online. 

Typical  Origins  of  MA  Requests 

Normally,  MA  requests  come  from  acquisition  organizations 
after  learning  more  about  the  extent  of  DAU's  capabilities  out¬ 
side  the  classroom,  either  through  DAU's  website,  or  from  a 


In  some  cases,  MA  without  a  deep  dive  is  analogous  to  ready, 
fire,  aim— in  other  words,  an  MA  event  without  aim  is  almost 
certain  to  miss  the  desired  target.  In  the  same  context,  DAU 
would  expend  resources  and  invariably  miss  the  target  without 
the  data  from  a  deep  dive. 

There  are  exceptions.  If  an  organization  has  a  shortage  of 
trained  personnel  to  meet  its  current  or  emerging  mission, 
DAU  can  respond  with  a  variety  of  time-urgent  training  op¬ 
tions.  However,  untrained  personnel  filling  certain  positions 
could  suggest  a  bigger  issue.  Why  are  they  filling  positions 
without  training?  Do  they  have  access  to  the  required  formal 
training,  or  has  the  organic,  on-the-job  training  program  waned 
or  become  ineffective?  DAU  uses  a  highly  interactive  work¬ 
shop  venue  in  which  our  customers  use  their  own  products, 
in  many  cases,  as  part  of  the  workshop.  That  is  the  perfect 
approach  for  newly  arrived  personnel  who  need  a  quick  course 
in  which  they  can  go  over  new  policies  or  refresh  their  skills. 
DAU  also  offers  various  workshops  that  can  be  conducted 
with  the  staff  or  workforce.  Some  examples  include  Acquisi¬ 
tion  Leadership  Development,  Stakeholder  Management,  Risk 
Management,  and  Services  Acquisition,  as  well  as  Continuous 
Learning  Modules  (CLMs)  and/or  modified  standard  course 
modules  tailored  to  the  organization  (ACQ  201B,  ENG  203  for 
Nuclear  weapons)  CLM  on  the  Nuclear  Phase  6.X  process. 
Some  actions,  policy  or  discipline,  for  example,  need  to  be 
owned  by  the  leadership,  while  workshops  with  specific  topics 
would  be  facilitated  by  DAU. 

A  deep  dive  that  uses  a  survey  usually  requires  a  series  of 
events  coordinated  with  the  organization  leadership. 

More  About  the  Survey  Process 

Following  a  series  of  conversations  with  various  leaders, 
DAU  begins  to  better  understand  the  customer's  organi¬ 
zational  dynamic.  But  sitting  down  with  the  customer  and 


Defense  AT&L:  January-February  2017 


42 


co-developing  an  in-depth  survey  with  specific  questions 
tuned  to  the  identified  challenges  is  a  sure  way  to  get  the 
data.  Designing  a  survey  is  not  as  simple  as  it  sounds.  It  re¬ 
quires  a  great  deal  of  preparation.  Survey  question  design 
is  not  a  task  for  one  person  but  rather  a  collaborative  effort 
between  DAU  and  the  customer.  During  these  question  de¬ 
sign  meetings,  DAU's  survey  team  works  directly  with  the 
customer  to  generate  questions  that  would  uncover  root 
causes  that  could  be  leading  to  the  wrong  kind  of  conse¬ 
quential  outcomes.  The  customer  is  involved  throughout 
the  development  period,  providing  feedback  on  the  draft 
survey.  DAU  can  sometimes  start  the  process  with  a  boil¬ 
erplate  survey  tailored  for  the  organization  requesting  MA. 

Rating  scale  questions  bundled  in  matrix-form  coupled  with 
open-ended  questions  provide  a  strong  qualitative  and  quan¬ 
titative  measurements  yielding  greater  credence  to  richer 
assessments.  These  deep  dive  surveys  can  also  serve  as  an 
organization's  health  check-up.  The  results  could  confirm  the 
need  to  provide  a  booster  shot  in  many  areas— including  com¬ 
munication,  trust,  conflict  management,  professional  develop¬ 
ment,  internal  processes,  feedback,  accountability,  advance¬ 
ments  and  recognition,  to  name  a  few  such  areas. 

Every  DAU  survey  also  includes  demographic  questions  in 
order  to  determine  how  different  populations  within  an  or¬ 
ganization  respond.  For  example,  a  large  percentage  of  the 
same  population  negatively  responding  to  the  same  question 
provides  is  a  valuable  data  point. 

Open-ended  questions  are  included  throughout  the  survey  to 
enable  respondents  to  open  up  about  new  areas  related  to  the 
topic  or  to  get  things  off  their  chests.  In  the  aggregate,  these 
may  result  in  new  themes  we  need  to  address. 

Survey  results  are  compiled  and  analyzed.  The  lead  faculty 
member  and  analyst  build  a  presentation  that  recommends 
course(s)  of  action  for  the  organization's  leadership. 

Protecting  the  respondents'  anonymity  is  of  the  utmost  impor¬ 
tance.  Every  survey  response  is  confidential  and  protected.  All 
survey  results  are  presented  in  aggregate,  and  the  requesting 
component  (organization)  never  sees  the  raw  data— another 
way  the  DAU  survey  team  protects  respondent  anonymity. 
Nothing  will  ever  be  presented  or  briefed  if  it  can  be  attributed 
to  any  individual  in  any  way. 

Many  types  of  formats  are  available  for  structuring  surveys, 
depending  on  the  desired  results  and  preference  of  the  devel¬ 
opers  and  analysts.  One  format  used  recently  focused  ques¬ 
tions  on  the  types  of  intervention,  corrective  action  or  MA 
that  were  available. 

Examples  (from  The  Performance  Consultant's  Field  Book:  Tools 
and  Techniques  for  Improving  Organizations  and  People  (2nd 
ed.).  John  Wiley  and  Sons,  2006.  [Figure  8.2.  The  Families  of 
Interventions]): 


DALTs  survey  team  works 
directly  with  the  customer 
to  generate  questions  that 
would  uncover  root  causes 
that  could  be  leading 
to  the  wrong  kind  of 
consequential  outcomes. 


■  Information  focused:  This  type  of  corrective  action  might 
be  as  simple  as  an  all-hands  meeting  to  disseminate  infor¬ 
mation  or  may  require  a  change  in  climate  that  would  foster 
better  communications.  Depending  on  other  related  factors, 
Crucial  Conversations,  Speed  of  Trust,  and  Stakeholder  re¬ 
lationship  workshops  might  also  be  appropriate. 

■  Consequences  focused:  If  poor  behavior  in  many  areas  is 
the  issue,  a  consequence-type  intervention  may  result.  This 
could  take  the  form  of  "New  Rules''  information  or  a  facili¬ 
tated  workshop  with  management  and  a  worker's  council  to 
develop  the  rules  and  appropriate  scaling  of  consequences. 

■  Design  focused:  These  activities  are  intended  to  change 
behavior,  beliefs  or  way  of  doing  business  by  modifying  the 
organizational  structure,  revising  the  reporting  chain,  or  by 
making  other  modifications  to  effect  the  desired  outcome. 
This  could  be  done  by  fiat  or  through  a  participatory  facili¬ 
tated  workshop. 

■  Capacity  and  capabilities  focused:  This  type  of  action 
might  be  appropriate  for  needed  training  because  of  new 
technologies,  new  workforce  members  or  changing  needs. 
With  the  workforce  diminishing  in  some  areas  and  with 
increasing  demand  for  support,  this  type  of  workshop  or 
activity  can  help  develop  an  optimal  solution  to  improve 
capacity  or  capability. 

■  Action  focused:  If  we  have  a  problem  and  the  commander 
wants  a  solution,  this  type  of  intervention  or  activity  might 
be  appropriate.  The  scope  could  range  from  a  simple  set  of 
directions  and/or  policy  proliferated  from  the  commander  to 
a  facilitated  workshop  with  appropriate  attendees  to  design 
the  actions  that  need  to  take  place. 

■  Congruence  focused:  This  is  indicated  when  we  need  to  get 
folks  on  the  "same  sheet  of  music"  so  everyone  is  working 
the  same  problem  at  the  same  time.  Over  time,  some  or¬ 
ganizations  drift  in  the  tight  control  and  discipline  required 
for  certain  activities.  A  congruence-focused  action  can  solve 
the  problem. 
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ACQuipedia 

Acquisition 

ENCYCLOPEDIA  OF 
COMMON  TERMS 

An  online  encyclopedia  that 
provides  the  acquisition  workforce 
with  quick  access  to  information 
on  common  acquisition  topics  and 
terms. 

Online  articles  provide  just  what 
you  need  to  know  in  a  succinct 
and  digestible  format: 

•  Definitions  and  narratives 

•  Links  to  related  policy,  guidance, 
lessons 
learned,  tools, 
communities, 
training,  and 
other  resources 


Your 
reference 
tool  for 
acquisition 
topics 

•  Quick 

•  Concise 

•  High-value 
content 

•  Searchable 

•  Available 
24/7— when 
and  where 
you  need  it 


https://DAP.DAU.MIL/ACQUIPEDIA 


Aside  from  the  quantitative  feedback  from  respondents  in 
deep  dives,  a  sampling  of  their  recommendations  includes: 

■  Reorganization 

■  All-hands  information  meetings 

■  Leadership  workshops 

■  Alternative  work  schedules 

■  Advancement  opportunities 

Following  the  deep  dive,  analysis  and  brief  to  the  command 
staff,  a  single  workshop  or  series  of  workshops  may  be  pre¬ 
sented.  In  most  cases,  the  more  compelling  imperative  is  that 
a  long-term  plan  be  implemented  to  help  meet  the  organiza¬ 
tion's  specific  needs,  coupled  with  longer-term  communica¬ 
tions  strategies  to  help  keep  command,  management  and 
working  staff  well  aligned. 

A  significantly  powerful  combination  involves  joining  Executive 
Coaching  and  the  related  Extraordinary  Future  envisioned  by 
the  coached  client  with  various  facilitated  workshops  to  get 
the  workforce  to  buy  in  and  achieve  that  Extraordinary  Future. 

DAU  continuously  seeks  customer  feedback  on  the  efficacy 
of  MA.  We  need  to  know  if  we  got  it  right,  good  or  bad— and 
what  changes  might  be  warranted  to  improve  our  products 
and  services. 

Using  the  results  of  the  survey,  an  organization's  leadership 
and  DAU  faculty  lead  collaborate  to  determine  a  long-term 
plan  for  the  organization.  With  Executive  Coaching,  Consult¬ 
ing,  the  Defense  Acquisition  Executive  Overview  Workshop, 
Soft  Skills  Leadership  training,  tailored  academic  modules,  and 
tailored  intervention  modules  such  as  Crucial  Conversations®, 
Will-Cost/Should-Cost,  or  cybersecurity  workshops,  the  op¬ 
tions  are  seemingly  limitless. 

What  DAU  Customers  Say  About  DAU  Support 

While  we  can't  name  the  organizations  for  which  we've  con¬ 
ducted  deep  dives,  here  is  what  one  organization  stated: 

The  initial  consultation  resulted  in  a  survey  which  helped  iden¬ 
tify  specific  problems  which  need  to  be  addressed.  The  second 
consultation  (the  classes)  revealed  an  interest  in  solving  the 
problems  that  went  beyond  just  the  managers.  Since  we  don't 
want  to  develop  any  solutions  until  everyone  has  been  involved, 
no  specific  metric  on  improvements  [is]  available,  but  as  previ¬ 
ously  said  word  has  spread,  and  the  workforce  is  hopeful  that 
viable  solutions  can  be  developed  and  implemented. 

While  a  deep  dive  might  sound  a  little  ominous,  the  organi¬ 
zational  gains  it  offers  are  well  worth  the  initial  investment.  It 
won't  fix  everything,  but  it  reinforces  what's  working  well  and 
what  needs  more  attention.  & 

The  authors  can  be  contacted  through  woody.spring@dau.mil. 


Defense  AT&L:  January-February  2017 


44 


Redefining  the  "Can  Do"  Attitude 

The  Language  of  a  Critical  Thinking  Culture 
Christopher  Phillips,  Ph.D.  ■  Bobbie  DeLeon,  Ed.D. 


ou  and  your  team  face  a  complex  problem.  What  is  your  first  response? 
What  are  your  thoughts  and  feelings  about  the  situation  or  words  that 
describe  the  issues? 


Y 

Take  the  word  "snow."  What  comes  to  mind?  What  thoughts  does  it  prompt?  Knowing  the  snow  is  of 
the  blizzard  variety  rather  than  a  mere  dusting  gives  you  better  clarity  and  precision  as  you  translate 
your  thoughts  into  words  and  then  deeds.  You  have  a  better  understanding  of  what's  taking  place,  and  what  action 
to  take,  whenever  you  have  a  more  precise  meaning. 


While  traipsing  through  northern  Canada  in  the  1880s,  noted  anthropologist  Franz  Boas  discovered  that  Alaska's 
Inuits  had  50  words  for  snow— words  like  "aqilokoq”  for  "softly  falling  snow"  and  "piegnartoq"  for  snow  that's  "good 
for  driving  sled."  There  is  little  wonder  they  had  an  array  of  nuanced  definitions  for  snow,  given  how  it  impacted 
virtually  every  aspect  of  their  lives. 


Language  shapes  our  thinking  and  our  culture.  It  shapes  our  lives  and  livelihoods,  how  we  go  about  our  day,  how  we 
make  decisions,  major  and  minor,  in  our  private  and  professional  lives.  In  our  professional  lives,  if  we  think  through 


Phillips  is  a  specialist  in  the  Socratic  method  of  group  learning  and  a  noted  speaker ;  consultant moderator  and  author.  DeLeon  is  a  professor 
at  the  Defense  Acquisition  University's  Defense  Systems  Management  College  at  Fort  Belvoir,  Virginia. 
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a  problem  using  a  systematic  process,  and  everyone  in  the 
group  knows  the  precise  meaning  of  the  words  used,  we  start 
with  a  strong  foundation  for  moving  toward  better  outcomes. 

What's  more,  if  we  ask  others  to  state  their  assumptions  or 
to  clarify  their  purposes,  we  have  a  common  language  from 
which  to  exchange  perspectives,  and  to  discover  whether 
they're  aligned  with  the  facts,  with  the  information  at  hand, 
and  the  conclusion  we've  drawn— or  not. 

Three  Important  Questions  in  a  “Thinking  Culture” 

■  What  are  my  assumptions? 

■  What  points  of  view  should  I  consider? 

■  What  is  the  purpose  of  my  thinking? 

This  thinking  language  advances  the  cause  of  critical  thinking 
itself  and  creates  a  thinking  culture. 

”Can  Do”  Attitude  and  Uncritical  Thinking 

How  about  the  term  "can-do"?  It  is  generally  used  to  charac¬ 
terize  leaders  and  teams  and  enjoys  a  positive  connotation. 
But  is  it  always  a  good  thing?  Does  that  mean  it  always  leads 
to  positive  outcomes? 

Vice  Admiral  Terry  J.  Benedict,  director  of  the  Navy's  Strategic 
Systems  Programs  (SPP)  was  determined  to  find  out.  Benedict 
and  his  staff  have  the  major  responsibility  of  nuclear  war  de¬ 
terrence.  Needless  to  say,  the  working  environment  is  fraught 
with  great  tension  and  risk,  with  little  margin  for  error.  The 
program  has  been  fulfilling  its  mission  for  60  years. 

But  to  Benedict,  who  took  over  in  2010,  SSP  can't  rest  on  its 
laurels.  The  stakes  are  too  high. 

The  year  after  he  assumed  the  helm  of  the  SSP,  the  Fukushima 
disaster  occurred  on  March  11,  2011.  A  massive  earthquake 
triggered  a  major  tsunami.  The  Fukushima  Daiichi  Nuclear 
Power  Plant  structures  were  not  capable  of  enduring  a  major 
tsunami,  nor  the  powerful  ground  motion  of  an  earthquake. 
These  events  disabled  the  power  supply  and  hence  the  ability 
to  cool  three  of  the  Fukushima  nuclear  reactors.  Their  cores 
melted  over  the  next  3  days. 

The  Fukushima  Analysis 

This  was  a  wake-up  call  for  Benedict.  He  pored  over  the  com¬ 
prehensive  report  that  described  the  chain  of  events  that  led 
to  the  wholesale  safety  breakdown  at  Fukushima.  The  report 
essentially  blamed  the  failure  to  prepare  against  earthquakes 
and  tsunamis  on  the  Japanese  culture  of  compliance  and  def¬ 
erence  to  authority  and  of  unilateral  control  in  the  decision¬ 
making  hierarchy. 

What  particularly  impacted  Benedict  was  the  report's  finding 
that  the  culture's  devotion  to  sticking  with  its  existing  safety 
program,  come  what  may,  its  reluctance  to  question  author¬ 
ity,  and  the  prevalent  "group  mentality"  all  contributed  to  the 
disaster.  These  characteristics  drove  how  decisions  were,  or 


were  not,  made  by  a  risk-ignorant  culture  that  worshiped  at 
the  altar  of  a  detrimental  sort  of  "can-do"  attitude. 

In  the  acquisition  workforce,  both  leaders  and  teams  often 
seek  to  cultivate  a  version  of  the  can-do  attitude.  In  the  case 
of  Benedict  and  the  SSP,  it  is  meant  to  be  done  in  a  deliberate 
way  that  ideally  leads  to  greater  safety  awareness. 

The  Lesson  of  Risk  Ignorance 

After  reading  the  detailed  accident  report  on  Fukushima, 
Benedict  was  prompted  to  ask  how  much  of  the  kind  of  mind¬ 
set  that  existed  among  employees  at  the  Fukushima  Daiichi 
Nuclear  Power  Plant  was  engrained  in  his  own  organization 
and  culture.  He  further  wondered  if  the  can-do  attitude  in  the 
SSP  culture  also  is  overdone.  Can  it  lead  to  the  same  kind  of 
risk  ignorance  as  it  did  in  Fukushima? 

One  principle  lesson  that  Benedict  gleaned  from  the  report 
along  with  a  greater  appreciation  of  Heinrich's  Law  was  this: 
If  you  ignore  all  the  little  or  so-called  minor  things  in  safety- 
mishaps,  accidents,  near  misses  and  oversights— and  a  big 
event  strikes  like  the  tsunami  that  triggered  a  cascade  of 
subsequent  catastrophes  at  Fukushima,  then  you  realize  in 
retrospect  that  it  was  the  little  things  that  had  been  hap¬ 
pening  all  along  that  led  to  the  tragedy.  The  tsunami  itself 
of  course  was  unavoidable,  but  much  of  what  unfolded  in  its 
wake  would  have  been  far  less  ruinous  if  there  had  been  a 
sound  critical  thinking  culture  surrounding  the  safety  pro¬ 
gram  at  the  power  plant. 

Benedict  understood  full  well  the  human  tendency  to  read  a 
report  like  this  and  conclude  after  the  fact,  "We  should  have 
seen  it  coming."  It  is  always  easy  to  see  such  things  with  per¬ 
fect  clarity  in  hindsight.  But  to  him,  it  was  necessary  to  have 
this  kind  of  clarity  of  insight— without  blind  spots— while  the 
situation  unfolds  in  real  time,  rather  than  afterward.  Ostensi¬ 
bly,  personnel  had  ignored  repeated  minor  mishaps  at  Fuku¬ 
shima  Daiichi  Nuclear  Power  Plant,  because  they  didn't  really 
grasp  their  potential  significance  and  therefore  didn't  respond 
appropriately.  The  staff  at  the  plant  did  not  possess  the  risk- 
aware  culture  that  Benedict  envisioned. 

Benedict  and  his  team  of  officers  sought  to  examine  other 
safety  calamities  that  had  root  causes  similarto  Fukushima— 
i.e.,  the  fatal  2012  attack  against  American  diplomats  in  Beng¬ 
hazi,  Libya,  as  well  as  the  USS  Greeneville  collision  and  the 
NASA  space  shuttle  disasters.  He  did  so  to  ensure  that  he  was 
promoting  at  the  SSP  a  risk-aware  rather  than  a  risk-ignorant 
culture.  The  culture  Benedict  pursued  was  one  in  which  all 
those  involved  in  the  enterprise  possessed  a  pervasive  willing¬ 
ness  to  rethink,  a  sense  of  accountability,  and  purposeful  cour¬ 
age— a  very  different  kind  of  "can-do"  attitude  than  the  one 
at  Fukushima  and  the  other  disasters  his  team  investigated. 

In  Benedict's  view,  success  is  a  lousy  teacher— and  so  the  60 
years  of  success  that  the  SSP  enjoyed  actually  served  as  even 
greater  impetus  for  him  to  "carry  it  forward"  and  to  make  sure 
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that  the  culture  he  was  developing  within  his  organization  was 
populated  with  critical  thinkers.  Yet  he  worried:  Are  we  just 
one  incident  away  from  Fukushima? 

His  team  also  studied  the  USS  Greeneville  disaster,  a  high- 
profile  case  in  which  a  U.S.  Navy  submarine  off  the  Japanese 
coast  surfaced  right  under  a  fishing  vessel,  killing  nine  of  the 
fishing  boat's  crew  members.  When  the  incident  was  inves¬ 
tigated,  it  was  discovered  that  chaos  had  been  the  norm  for 
that  submarine's  culture.  What  connected  Fukushima  with 
the  Greeneville  accident  was  that  personnel  in  both  cases  were 
supremely  risk  ignorant  and  also,  to  a  lesser  extent,  risk  averse. 
Their  culture  was  characterized  by  the  pervasive  behavior  of 
rushing  through  safety  procedures,  with  careless  accommo¬ 
dation  providing  the  norm,  rather  than  operating  within  the 
boundaries  of  good  practice  and  normal  protocol— very  much 
the  wrong  kind  of  can-do  attitude. 

When  Benedict's  team  studied  both  the  Challenger  and  Colum¬ 
bia  space  shuttle  disasters  they  also  found  a  deleterious  kind 
of  "can-do”  attitude  inspired  by  past  successes.  This  created  a 
kind  of  groupthinkthat  discouraged  individuals  from  stepping 
up  and  questioning  flawed  safety  practices. 

Elements  of  a  Strategic  Systems  Program 

■  Deliberately  designs  a  risk  aware/risk  evaluation  culture 

■  Promotes  a  questioning  attitude 

■  Encourages  ideas  and  criticism 

■  Has  transparent  decision  support 

■  Has  rigor  and  open  self-appraisal 

■  Practices  humility  and  leadership  by  example 

Risk  Aware  Thinking — a  Deliberate  Design 

To  Benedict,  it  was  of  paramount  importance  to  deliberately 
design  a  risk-aware  culture— because  he  was  keenly  aware 
that  if  you  don't  design  a  culture  yourself,  one  will  be  created  in 
the  vacuum,  and  quite  possibly  be  of  the  risk-ignorant  variety. 

The  culture  that  Benedict  set  about  creating  after  pondering 
the  in-depth  comparative  studies  by  his  staff  was  one  that 
encouraged  the  continual  generation  of  new  ideas  related  to 
safety,  and  that  valued  above  all  else  a  questioning  attitude- 
vital  attributes  for  a  thinking  culture. 
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Fukushima,  and  the  lessons  to  be  learned  that  relate  to  the 
SSP's  own  mission. 

Furthermore,  new  employees  are  issued  a  card  that  enumer¬ 
ates  essential  human  traits  and  mitigating  risk-aware  behav¬ 
iors,  along  with  an  explanation  of  why  it  is  vital  for  achieving 
the  SSP's  one-of-a-kind  mission.  As  a  member  of  the  SSP, 
employees  also  must  be  able  to  explain  the  reasoning  behind 
a  recommended  decision.  What's  more,  the  decision-making 
process  itself  is  extremely  transparent.  As  a  consequence, 
there  is  no  unilateral  authority,  and  so  no  single  person  has 
the  power  to  make  a  decision— a  bedrock  component  of 
Benedict's  culture  of  deliberate  design.  Additionally,  there 
are  numerous  checks  and  balances  among  the  different 
groups  in  this  "flat  organization”  that  has  little  hierarchy.  All 
SSP  staff  work  together  and  interact  as  part  of  an  integrated 
whole  with  a  shared  sense  of  mission  and  purpose  in  this 
high-consequence,  high-tension  environment. 

Moreover,  as  part  of  the  overarching  goal  of  creating  a  risk- 
aware  culture,  everyone  at  the  SSP  has  the  right  to  ask,  and  is 
encouraged  to  ask,  "Why  are  you  doing  this?”  Staff  members 
are  given  considerable  autonomy  intermingled  with  regular 
feedback  from  peers.  This  keeps  everyone  on  track  and  on 
board.  Those  who  perform  exceptionally  well  in  promoting 
and  promulgating  this  risk-aware  kind  of  can-do  culture  are 
rewarded  and  recognized— not  just  those  within  the  gov¬ 
ernment  who  work  directly  for  the  SSP,  but  their  industry 
partners  as  well. 


Attributes  of  a  “ Thinking  Culture" 

■  Establishes  critical  thinking  as  a  habit 

■  Clarifies  thinking  and  rationale 

■  Involves  thinking  and  collaboration  with  others 

■  Uses  deliberate  practice,  evaluation  and  feedback  to  real¬ 
ize  improvement 

Benedict  had  in  effect  deliberately  designed  a  thinking  cul¬ 
ture.  In  order  to  make  critical  thinking  an  ingrained  habit,  he 
established  a  protocol  in  which  everyone  who  joins  the  SSP  is 
made  aware  from  the  get-go  that  the  goal  is  to  strive  relent¬ 
lessly  to  be  risk  aware.  New  personnel  orientation  includes 


Frank  Kendall,  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics,  has  stressed  that  critical  thinking 
is  "necessary  for  success”  and  that  it  means  "figuring  out  the 
best  course  of  action  in  a  specific  circumstance,  balancing 
all  of  the  complex  factors  that  apply  to  a  given  situation.” 
He  could  well  have  been  describing  the  culture  of  Benedict's 
program  and  the  crucible  in  which  thoughtful  decisions  are 
made  every  day  so  that  the  SSP  continues  successfully  with 
its  mission.  SSP  truly  is  characterized  by  a  critical  thinking, 
can-do  culture.  & 

The  authors  can  be  contacted  at  bobbie.deleon@dau.mil  and  at 
christopher_phillips@mac.com. 
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Auditing  Organizational  Security 


Eugene  A.  Razzetti 


ing  organizational  security  is  no  different  from 
gin g  any  other  of  the  command's  missions. 
Establish  your  policies,  goals  and  risk  parameters; 
implement,  train,  measure  and  benchmark  them. 
And  then  audit,  audit,  audit. 


Today,  more  than  ever,  Organizational  Security  is  an  essential  component  of  a  robust, 
responsive  military  command.  And  commands  that  cannot  execute  their  operations  in  a 
self-imposed  and  self-monitored  secure  environment  may,  at  best,  cease  to  be  effective  or, 
at  worst,  cease  to  exist.  This  is  the  same,  certain  fate  that  befalls  private  enterprises  that 
cannot  maintain  operational  effectiveness,  profitability  or  product  superiority— except  it 
happens  faster  in  the  private  sector. 

Organizations  must  harden  their  operations  to  protect  them  from  either  incidental  or 
deliberate  attack.  Internal  (or  self-)  auditing  is  essential  to  the  hardening  process. 

Cybersecurity,  the  concept  most  frequently  promoted  these  days,  is  a  body  of  technolo¬ 
gies,  processes  and  practices  designed  to  protect  networks,  computers,  programs  and 
data  from  attack,  damage  or  unauthorized  access.  Is  cybersecurity  important  and  nec¬ 
essary?  Of  course!  However,  cybersecurity  should  not  be  regarded  as  independent  or 
standing  alone.  Cybersecurity  is  an  indispensable  element  of  organizational  security,  which 
is  the  subject  of  this  article. 

Figure  1  describes  the  many  organizational  security-related  challenges  that  military 
commands  (including  cybersecurity)  confront  in  moving  from  planning  to  executing 
their  missions. 


Razzetti,  a  retired  US.  Novy  captain ,  is  a  management  consultant  military  analyst  and  certification  auditor. 
He  is  the  author  of  five  management  books ,  numerous  articles  and  analytical  reports ,  and  has  served  on  the 
advisory  boards  of  two  business  schools. 
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Figure  1.  The  Big  Picture:  Organizational  Security  in 
Mission  Execution 
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establish  and  maintain 
an  ongoing  robust  se¬ 
curity  posture.  This  in¬ 
volves  eternally  raising 
the  barand  leaving  the 
current  status  quo  in 
the  rearview  mirror.  I 
recommend  that  com¬ 
manders  who  want  to 
establish  and  maintain 
structured  informa¬ 
tion  systems  security 
management  review 
the  following  from  the 
International  Organi¬ 
zation  for  Standardiza¬ 
tion  (ISO):  ISO  27000: 
Information  Systems  Se¬ 
curity  Management. 


AT/FP:  Anti-terrorism/Force  protection 
MOE:  Measures  of  Effectiveness 


Several  years  ago,  I  worked  as  a  military  analyst  on  programs 
that  included  information  warfare  (like  all  modern  defense  pro¬ 
grams).  The  lesson  I  continually  relearned  during  that  time  was 
that  information  is  the  only  "weapon”  that  can  be  in  more  than 
one  place  at  the  same  time. 

As  information  technology  (IT)  is  increasingly  integrated 
with  physical  infrastructure,  the  risk  increases  of  wide-scale 
or  high-consequence  events  that  could  harm  or  disrupt  mili¬ 
tary  commands  and  their  missions.  Therefore,  strengthening 
organizational  security  and  resilience  is  critical. 


A  robust  program  of 
internal  auditing  of  a 
command's  organiza¬ 
tional  security  hardens 
and  protects  military 
operations  under  a 
structured  organiza¬ 
tional  security  management  system.  Anything  less  than  robust 
jeopardizes  the  existence  of  the  command,  the  capability  of  its 
leadership  and  the  fulfillment  of  its  missions. 

There  are  10  auditable  areas  in  which  commands  can  create 
and  sustain  credible,  effective  and  secure  management  sys¬ 
tems  and  strategies— for  headquarters  commands,  subordi¬ 
nates  in  the  field  and  suppliers. 

1.  Policy  Development 

Commanders  must  develop,  as  applicable  to  the  mission,  writ¬ 
ten  security  policies  that  are: 


All  U.S.  military  commands  depend  on  IT  systems  and 
computer  networks  for  essential  operations  and  mission 
fulfillment.  IT  systems  face  large  and  diverse  cyber  threats 
that  range  from  unsophisticated  hackers  to  technically  com¬ 
petent  intruders  using  state-of-the-art  intrusion  techniques. 
Many  malicious  attacks  are  designed  to  steal  information 
and  disrupt,  deny  access  to,  degrade  or  destroy  critical 
information  systems  or  to  put  infrastructure  (e.g.,  power 
plants)  out  of  commission. 

Internal  and  external  auditing  of  organizational  security 
programs  can  ensure  compliance  with  requirements  and 
can  sustain  an  acceptable  level  of  impregnability.  However, 
generating  preventive  and  corrective  actions  as  a  result  of 
those  audits  and  reassessing  goals  and  objectives  based 


Consistent  with  the  other  policies  of  the  organization  and 
those  of  higher  authority 

Specifically  keyed  to  planned  security  objectives,  targets, 
and  programs 

Consistent  with  the  organization's  overall  security  threat 
and  risk  management  strategy  and  the  nature  and  scale  of 
its  operations 

Clear  in  stating  overall/broad  security  management 
objectives 

Documented,  implemented  and  monitored 
Communicated  to  all  levels  and  to  third  parties,  including 
contractors  and  visitors,  so  that  they  all  are  made  aware 
of  their  security-related  obligations. 
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Things  refuse  to  be 
mismanaged  long. 

—Ralph  Waldo  Emerson 

2.  Program  Management 

Effectively  managing  any  program  requires  the  continual 
monitoring  of  the  effectiveness  of  projects,  procurements 
and  suppliers,  establishment  of  metrics  and  early  identifica¬ 
tion  of  potential  problems.  Commands  must  assess  all  their 
functions  and  spend  their  limited  resources  according  to  how 
much  their  vulnerability  is  reduced  by  that  expenditure,  as 
shown  in  Figure  2. 

As  the  arrows  suggest,  managers  want  to  minimize  funds 
committed  to  ineffective  programs.  The  goal  of  the  program 
management  (with  programs  pictured  as  small  pyramids)  is  to 
move  programs  into  Quadrants  II  and  III.  Programs  in  Quad¬ 
rant  I  may  appear  acceptable  but  can  breed  complacency,  and 
there  is  no  longer  any  room  for  complacency  in  organizational 
security.  Programs  or  projects  that  fall  into  Quadrant  IV  are  un¬ 
acceptable  and  require  forthright  (and  probably  unwelcome) 
corrective  action. 

At  the  same  time,  commanders  must  establish  program 
management  roles,  responsibilities  and  authorities  that  are 
consistent  with  achieving  security  management  policies  and 
objectives.  And  these  must  be  communicated  to  all  respon¬ 
sible  parties. 

Commanders  need  to  make  a  commitment,  measurably  and 
consistently,  to  developing  a  Security  Management  System 
(SMS)  and  continually  improving  its  effectiveness.  This  is  ac¬ 
complished  specifically  by: 


3.  Security  Risk  Management 

Security  risk  management,  like  any  other  focused  risk  manage¬ 
ment  strategy,  requires  that  commanders  identify  and  assess 
"risk”  in  terms  of  threats,  criticalities  and  vulnerabilities  to  the 
commands  and  their  assigned  missions.  Commanders  must 
establish  and  maintain  strategies  for  the  ongoing  identification, 
assessment  and  mitigation  of  all  risks,  especially  those  related 
to  organizational  security.  Mitigation  means  identifying  and 
implementing  effective  control  measures.  In  the  execution  of 
control  measures,  risk  assessment  becomes  risk  management. 
An  effective  security  risk  assessment  strategy  should  include 
identifying  (when  appropriate): 

■  Physical  failure  threats  and  risks,  such  as  functional  fail¬ 
ure,  incidental  damage,  malicious  damage  or  terrorist  or 
criminal  action 

■  Operational  threats  and  risks,  including  the  control  of 
security,  human  factors  and  other  activities  that  affect  the 
organization's  performance,  condition  or  safety 

■  Factors  outside  of  the  organization's  control  such  as  fail¬ 
ures  in  externally  supplied  (e.g.,  outsourced)  equipment 
and  services 

■  Security  equipment,  including  replacement,  maintenance, 
information  and  data  management  and  communications 

■  Any  other  threats  to  the  continuity  of  operations 

Please  see  my  article:  "Robust,  Replicable  and  Defensible  Risk 
Management— At  Headquarters  or  the  Front”  in  the  July- 
August  2016  issue  of  Defense  AT&L  magazine. 

4.  Security  Training  and  Qualification 

Security-minded  organizations  appoint  (and  entrust)  person¬ 
nel  to  operate  their  security  management  systems.  Like  any 
other  responsible  positions  in  the  military,  the  people  who 
design,  operate  and  manage  the  security  equipment  and 
processes  must  be  suitably  qualified  in  education,  training, 


■  Communicating  to  all  parts  of  the  or¬ 
ganization  the  importance  of  meeting 
security  management  requirements 
in  order  to  comply  with  established 
policies 

■  Ensuring  any  security  programs  gener¬ 
ated  from  other  parts  of  the  organiza¬ 
tion  complement  the  security  manage¬ 
ment  system 

■  Establishing  meaningful  security  met¬ 
rics  and  measures  of  effectiveness 

■  Ensuring  security-related  threats,  criti¬ 
calities  and  vulnerabilities  are  evalu¬ 
ated  and  included  in  organizational  risk 
assessments  where  appropriate 

■  Ensuring  the  viability  of  the  security 
management  objectives,  targets  and 
programs. 


Figure  2.  A  Cost  vs.  Effectiveness  Matrix  (Example) 
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certification  and/or  experience.  I  put  the  word  "qualified"  in 
italics  because  training  may  not  be  enough.  Commanders  need 
qualification  programs— not  just  a  training  plan— for  all  critical 
positions  and  watch  stations. 

Furthermore,  all  personnel  must  be  fully  aware  and  supportive 
of  the  importance  of  compliance  with  security  management 


curity  management  policy,  objectives,  targets  or  programs, 
processes  or  procedures,  and  the  introduction  of  new  se¬ 
curity  infrastructure,  equipment,  or  technology  also  should 
be  documented. 

Auditing  the  supply  chain  also  means  auditing  compliance  with 
legal,  statutory  and  other  regulatory  security  requirements, 


The  security-minded  organization 
needs  to  establish,  implement  and  maintain 
appropriate  plans  and  procedures  (including 
creating  back-up  records  or  files)  for  responses 
to  security  breaches  and  emergencies  and  to 
prevent  and/or  mitigate  likely  consequences. 


policies  and  procedures  and  of  the  requirements  of  the  Secu¬ 
rity  Management  System,  as  well  as  their  own  roles  in  achiev¬ 
ing  compliance.  This  includes  emergency  preparedness  and 
response,  and  awareness  of  the  potential  security  implications 
of  deviating  from  specified  procedures. 

5.  Supply  Chain  Security 

Every  military  organization  has  a  supply  chain.  Security  re¬ 
quirements  and  attendant  risks,  whether  upstream  or  down¬ 
stream  of  its  activities,  can  profoundly  affect  operations,  prod¬ 
ucts  or  services.  Identifying,  evaluating  and  mitigating  threats 
posed  from  upstream  or  downstream  supply  chain  activities 
is  just  as  important  as  it  is  for  performing  the  same  functions 
inside  your  own  "fence  line." 

Commanders  would  do  well  to  audit  outside  that  fence  line. 
They  can  do  so  by: 

■  Identifying  all  links/nodes  of  the  supply  chain  and  ensur¬ 
ing  they  conform  to  stated  security  management  policies, 
controls,  and  mitigation  of  unacceptable  risks 

■  Examining  documented  procedures  for  situations  in  which 
a  lack  of  procedures  could  lead  to  failure  to  maintain 
operations 

■  Establishing  the  security  requirements  for  contractor- 
furnished  goods  or  services  that  impact  mission 
accomplishment 

■  Providing  hardened  and  redundant  lines  of  communication 

Where  existing  designs,  installations  or  operations  are 
changed,  documentation  should  address  attendant  revi¬ 
sions  to  command  structure,  roles  or  responsibilities.  Se- 


security  management  objectives,  delivery  of  security  man¬ 
agement  programs,  and  whether  the  program  provides  the 
required  level  of  security  (convoys,  containers,  warehouses, 
etc.).  In  my  experience,  there  can  be  no  control  of  the  supply 
chain  without  a  viable  and  robust  auditing  function. 

6.  Communication  and  Documentation 

Commands  must  have  secure,  hardened  and  redundant  proce¬ 
dures  for  disseminating  all  pertinent  security  management  in¬ 
formation.  This  applies  to  outsourced  or  host  nation-provided 
operations  as  well  as  those  taking  place  within  the  organiza¬ 
tion.  This  is  especially  important  when  dealing  with  sensitive 
or  classified  information. 

A  security  management  system  documentation  system  in¬ 
cludes  but  is  not  limited  to: 

■  The  security  management  system  scope,  policy,  objec¬ 
tives  and  targets 

■  Description  of  the  main  components  of  the  security  man¬ 
agement  system  and  their  interaction,  with  reference  to 
related  documents 

■  Documents  such  as  records  the  organization  determines 
to  be  a  necessary  part  of  ensuring  the  effective  planning, 
operation  and  control  of  processes  related  to  its  signifi¬ 
cant  security  risks. 

7.  Emergency  Preparedness  and  Response 

Emergency  response  may  be  thought  of  as  conducting  nor¬ 
mal  operations  at  faster-than-normal  speeds— or  something 
entirely  different.  The  security-minded  organization  needs 
to  establish,  implement  and  maintain  appropriate  plans  and 
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procedures  (including  creating  back-up  records  or  files)  for 
responses  to  security  breaches  and  emergencies  and  to  pre¬ 
vent  and/or  mitigate  likely  consequences. 

Auditing  emergency  plans  and  procedures  should  include 
all  reviewing  (and  any  testing)  information  that  may  be  re¬ 
quired  for  identified  facilities  or  services  during  or  after  in¬ 
cidents  or  emergencies  in  order  to  maintain  continuity.  The 
best  emergency  planning  I  ever  saw  was  at  U.S.  Navy  Bases 
along  the  Gulf  Coast,  which  face  an  immense  and  perennial 
threat  from  hurricanes.  Commanders  and  staff  members 
periodically  should  "stress-test”  the  effectiveness  of  their 
emergency  preparedness,  response  and  recovery  plans  and 
procedures,  especially  after  incidents  or  emergencies  caused 
by  security  breaches  and  threats.  They  should  test  these  pro¬ 
cedures  periodically. 

A  supporting  program  of  internal  or  outside  security  audits 
also  confirms  whether  the  organization  is  complying  with  rele¬ 
vant  legislation  and  regulations,  best  practices  and  the  policies 
and  objectives  established  by  higher  authorities.  Commands 
need  to  maintain  records  of  results,  findings  and  required  pre¬ 
ventive  and  corrective  actions. 

Security-minded  commanders  and  staffs  can  audit  their  secu¬ 
rity  management  plans,  procedures  and  capabilities.  Security 
audits  can  include  periodic  reviews,  testing,  post-incident  re¬ 
ports  and  lessons  learned,  performance  evaluations  and  ex¬ 
ercises.  Significant  findings  and  observations,  once  properly 
evaluated  or  gamed,  should  be  reflected  in  revisions  or  modi¬ 
fications  of  policies  and  procedures. 

8.  Daily  “Quick  Looks” 

Here  are  some  immediate  feedback  operational  initiatives  for 
forward-thinking  and  security-minded  organizations  trying 
to  identify  and  mitigate  (on  a  daily  basis)  their  vulnerability 
to  exploitation.  Develop  some  checklists,  and  "check  out” 
the  following: 

■  Intrusion  detection  systems 

■  Fences,  security  lighting,  natural  barriers 

■  Closed-circuit  TV 

■  Computer  backup  systems;  "firewalls”  against  viruses  and 
intrusions 

■  Roof  and  ventilation  duct  accessibility 

■  Construction  materials  and  thickness  requirements 

■  Installed  firefighting  systems 

■  Roads,  alleys  and  storm  drains 

■  Parking  areas 

■  Sewage  treatment  systems 

■  Locks,  doors  and  access  control 

■  Identification  management  (i.e.,  employees,  customers 
and  vendors) 

■  Utilities  (including  uninterruptible  power  systems  and 
emergency  generators) 

■  Safes,  desks,  filing  cabinets,  controlled/exclusion  areas 

■  Hazardous  materials  generation,  storage,  and  management 


■  Vehicle  surveillance  and  security  (including  delivery  and 
fuel  trucks) 

■  Proximity  of  emergency  services  (i.e.,  fire  departments, 
medical  emergency  services,  and  police) 

■  Mail  and  package  processing 

9.  Preventive  and  Corrective  Action 

Audit^Nonconformity^P/C  Action^Corrected/Improved 

Auditors  (by  any  name)  discover  "nonconformities.”  They 
identify  the  need  for  either  preventive  or  corrective  action. 
Top  management  (we  hope)  supports  the  audit  findings  and 
initiates  preventive  or  corrective  actions  and  seeks  feedback 
and  follow-up  to  measure  the  success  (or  lack  thereof)  of 
these  actions. 

Audits  of  organizational  security  are  no  different  than  audits  of 
any  other  management  program.  In  fact,  the  need  for  prompt 
corrective  action  may  be  even  more  critical. 

10.  Continual  Improvement 

Continual  improvement  is  the  basis  and  underpinning  of  the 
ISO.  All  processes  must  be  considered  ongoing  and  never  at 
an  "end  state.”  Top  management  develops  a  continuous  im¬ 
provement  mindset  that  something  can  always  be  improved. 
Continual  improvement  of  organizational  security  requires  that 
commanders  and  staffs  review  their  security  management 
systems  at  planned  and  frequent  intervals.  This  is  necessary 
in  order  to  ensure  continuing  effectiveness  in  an  ever-changing 
environment.  Security  audits  and  reviews  should  include  as¬ 
sessing  opportunities  for  improvement  and  the  attendant  need 
to  revise  the  security  management  system,  including  security 
policies  and  security  objectives,  plus  threats  and  risks.  Orga¬ 
nizations  already  working  with  ISO  9000  and  IS0 14000  can, 
with  minimal  effort,  expand  internal  audits  and  management 
reviews  to  cover  security  and  well  as  quality  and  environmental 
management.  See  the  American  Society  for  Quality  website 
at  www.asq.org. 

Summary 

Information  can  be  exploited  in  many  ways,  and  auditing 
organizational  security  has  tremendous  potential  for  experi¬ 
enced  commanders  and  staffs  to  harden  their  resources  and 
missions.  The  opportunities  for  continual  improvement  from 
auditing  are  as  vast  as  cyberspace  and  as  identifiable  as  of¬ 
fice  furniture. 

Organizational  security  must  be  part  of  every  mission.  Outputs 
from  security  audits  should  be  the  catalyst  for  any  revisions  to 
the  security  management  system,  together  with  cost-benefit 
analyses,  schedules,  risk  revisions,  and  other  justifications. 
Establish  policies  and  procedures,  identify  threats,  conduct 
risk  assessments,  implement  processes,  identify  corrective 
actions,  and  establish  a  mindset  of  continual  improvement. 
And  audit.  & 
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OSD  Logistics  Fellowship— A  View  From  Above 

Bryan  L.  Jerkatis 


A  friend  and  retired  U.S.  Air  Force  Command  Chief 
would  often  use  an  analogy  with  young  troops  re¬ 
garding  the  differences  in  their  world  views  versus 
those  of  their  leadership.  "Your  view  of  the  ground 
(truth)  depends  upon  the  height  of  the  branch  in  the 
tree  upon  which  you  are  standing  ... ,"  he  would  explain. 

Nearly  3  million  men  and  women  make  up  the  Department 
of  Defense  (DoD).  How  few  truly  have  an  opportunity  to  gain 
insight  and  understanding  of  the  origins  of  legislation,  budget, 
policy  and  oversight?  The  Office  of  the  Secretary  of  Defense's 
(OSD)  Logistics  Fellowship  provides  selected  logisticians  the 
opportunity  to  broaden  their  perspectives  and  consider  other 
points  of  view. 

The  OSD  Logistics  Fellows  Program  is  open  to  field  grade 
officer  (04-05)  and  DoD  civilian  equivalent  (GS  13-14) 
logisticians.  This  1-year,  unit-funded  "fellowship”  is  a  de¬ 
velopmental  assignment,  with  a  goal  of  providing  an  at¬ 
mosphere  that  fosters  learning,  growth,  and  experiential 
opportunities.  The  program  is  administered  by  the  Office 
of  the  Assistant  Secretary  of  Defense  for  Logistics  and  Ma¬ 
teriel  Readiness  (ASD[L&MRJ).  Fellows  have  the  unique 
opportunity  to  participate  in  policy  formulation  and  DoD- 
wide  oversight  responsibilities. 

Fellows  are  fortunate  to  travel  and  tour  both  the  public  and 
private  sectors  in  order  to  observe,  contrast  and  learn  first¬ 
hand  how  logistics  operations  compare  in  private  industry  and 
then  benchmark  best  practices.  Fellows,  through  visits  to  Con¬ 
gress,  gain  exposure  and  insight  into  the  legislative  processes. 
They're  also  able  to  attend  national  forums  and  engage  in  col¬ 
laborative  efforts  with  industry  partners.  Depending  on  their 
assignments,  fellows  may  have  the  opportunity  to  visit  and 
become  familiar  with  other  government  agencies,  as  well.  Per¬ 
haps  even  more  important,  the  fellowship  affords  participants 
opportunities  to  observe  and  interact  with  both  appointed  and 
career  senior  executives,  and  flag  officers— including,  "one- 


on-one”  meetings  with  senior  logistics  leaders  in  the  military 
departments,  Joint  Staff,  OSD  and  agencies. 

The  insights  and  "big  picture”  knowledge  to  be  gained  are  vir¬ 
tually  endless,  and  the  fellows  determine  much  of  their  own 
training  and  class  agendas.  During  my  fellowship,  I  was  tasked 
to  resolve  a  longstanding  logistics  policy  challenge  within  the 
DoD  and  was  given  considerable  leeway  to  gain  needed  ex¬ 
pertise  and  formulate  a  recommendation.  That  led  to  pub¬ 
lication  of  a  new  DoD  Directive.  Other  fellows  led  financial 
accountability  program  initiatives,  participated  in  DoD-level 
awards  processes,  led  worldwide  maintenance  symposiums, 
participated  in  source-selection  committees  and  a  number  of 
other  DoD-level  initiatives. 

Finally,  there's  a  fellowship  component  of  the  OSD  Logistics 
Fellowship  Program.  Fellows  share  a  common  bond,  form  a 
support  structure  and  face  many  diverse  challenges  together. 
The  OSD  Logistics  Fellows  Program  provides  an  opportunity 
to  forge  lifelong  bonds  and  friendships  with  other  logistics 
professionals  and  build  networking  capabilities  that  will  serve 
them  for  the  remainder  of  their  careers  and  beyond.  Upon 
completion,  fellows  return  to  their  sponsoring  organizations 
or  follow-on  assignments  with  increased  management  skills, 
technical  expertise  and  networks  that  span  DoD  logistics. 

The  OSD  Logistics  Fellowship  Program  provides  DoD  lo¬ 
gisticians  not  only  a  rich  experiential  odyssey  but,  perhaps 
more  important,  the  opportunity  to  obtain  a  deeper  career 
understanding  of  the  OSD  perspective  and  how  it  affects 
the  DoD  enterprise. 

NOTE:  The  selection  process  for  the  2017-2018  OSD  Logistics  Fellow¬ 
ship  Program  begins  in  January,  per  the  Office  of  the  ASD(L&MR). 
More  information  can  be  found  at  http://www.acq.osd.mil/log/LMR/ 
fellows_program.html.  & 


Jerkatis  was  a  member  of  the  OSD  Logistics  Fellows  Class  of  2015-2016. 
The  author  can  be  contacted  at  bryan.jerkatis@us.af.mil. 
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